feat(extensions): neutral behavior vocabulary with per-agent translation, fix LLM invocability by mbachorik · Pull Request #2103 · github/spec-kit

mbachorik · 2026-04-06T09:50:19Z

Context

When spec-kit commands were moved to the SKILLS format, the SKILL.md renderer hard-coded disable-model-invocation: true for every built-in command. This silently broke the ability for LLMs and orchestrating agents (Claude, Copilot, etc.) to invoke spec-kit commands — only humans could trigger them via slash commands.

At the same time, extension authors had no portable way to control agent behavior (model, tools, execution mode) without writing agent-specific frontmatter — meaning the same extension couldn't be written once and deployed to Claude Code, GitHub Copilot, and Codex with consistent behavior.

This PR addresses both issues together, as they share the same root: the need for a neutral behavior vocabulary that translates to correct per-agent frontmatter at render time.

What

Fix disable-model-invocation regression — built-in spec-kit commands now declare behavior: invocation: automatic in their source frontmatter, which translates to disable-model-invocation: false in Claude SKILL.md files, restoring LLM invocability.

Neutral behavior: vocabulary — extension authors declare intent once using neutral keys; the renderer translates them to agent-specific frontmatter:

behavior key	values	example agent mapping (Claude)
`invocation`	`explicit` / `automatic`	`disable-model-invocation: true/false`
`execution`	`command` / `isolated` / `agent`	`context: fork` (isolated)
`capability`	`fast` / `balanced` / `strong`	`model: claude-haiku…` / `claude-sonnet…` / `claude-opus…`
`effort`	`low` / `medium` / `high` / `max`	`thinking:` budget
`tools`	`none` / `read-only` / `write` / `full` / custom list	`tools: […]`
`visibility`	`user` / `model` / `both`	`type: prompt` vs `type: background`
`color`	`red` / `blue` / …	`color:` (Claude Code UI)

Agent-specific escape hatch — agents: claude: … / agents: copilot: … blocks in source frontmatter pass through agent-specific keys that have no neutral equivalent (e.g., argument-hint, handoffs).
execution: agent routing — commands with behavior: execution: agent are deployed to .claude/agents/ (Claude sub-agents) and Copilot .agent.md files, with mode and tools injected automatically.
Extension-relative path rewriting (original fix from fix: Extension SKILL.md files contain unresolvable relative paths to extension subdirectories #2101) — bare relative paths in extension command bodies (e.g., agents/control/commander.md) are rewritten to .specify/extensions/<id>/… in generated SKILL.md files, so AI agents can resolve them from the workspace root.

Why it is backwards compatible

Extensions without a behavior: block continue to work exactly as before — the translator is a no-op when the key is absent.
Agent-specific frontmatter keys already present in source files (disable-model-invocation, context, model, etc.) are passed through unchanged unless a behavior: block also exists.
Path rewriting only activates for directories containing extension.yml; preset directories are skipped.

Testing

Tested primarily in GitHub Copilot and Claude Code with a real extension install.

Unit tests added:

tests/test_behavior_translator.py — all neutral → agent-specific translations
tests/test_agent_deployment.py — execution: agent routing to .claude/agents/ and .agent.md
tests/test_extension_skills.py — agent-specific passthrough, behavior merging, backwards compat
tests/test_extensions.py — extension path rewriting (core fix, kimi, aliases, no-subdirs)
tests/integrations/test_integration_claude.py — end-to-end Claude integration

Opt-in integration test:

tests/test_integration_extension_skill_paths.py — installs a real extension and validates all generated skill files end-to-end (requires SPECKIT_TEST_EXT_DIR env var)

🤖 Generated with Claude Code

Copilot

Pull request overview

This PR fixes extension command bodies that reference extension-root-relative files (e.g., agents/..., knowledge-base/...) by rewriting those references to the installed location under .specify/extensions/<id>/... when generating SKILL.md files for skill-format agents (notably codex and kimi).

Changes:

Add CommandRegistrar.rewrite_extension_paths() and invoke it during SKILL.md rendering (before placeholder resolution).
Thread source_dir through SKILL.md generation for primary commands and aliases so the renderer has enough context to rewrite paths.
Add unit tests (extensions) and an opt-in integration test (real extension install) to validate end-to-end behavior.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
`src/specify_cli/agents.py`	Adds extension-relative path rewriting and applies it during SKILL.md rendering; threads `source_dir` into SKILL.md generation paths.
`tests/test_extensions.py`	Adds unit tests covering extension path rewriting for codex/kimi, aliases, and conservative “no subdirs” behavior.
`tests/test_integration_extension_skill_paths.py`	Adds opt-in integration tests that install a real extension and validate rewritten paths across generated skill files.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mbachorik · 2026-04-07T10:14:45Z

Fixed the path-rewriting scope issue raised in the inline review.

Change: Gated rewrite_extension_paths() in render_skill_command() on (source_dir / "extension.yml").exists(). This ensures the rewrite only applies to extension source directories; preset directories (which have preset.yml but no extension.yml) are skipped entirely.

Test added: test_preset_skill_registration_does_not_rewrite_paths — sets up a preset-style dir with a templates/ subdir, registers commands, and asserts the output contains no .specify/extensions/... paths.

Copilot

Pull request overview

This PR fixes extension command SKILL.md generation so that references to extension-relative files (e.g., agents/..., templates/...) are rewritten to their installed location under .specify/extensions/<id>/..., making those files resolvable by skill-format agents (Codex/Kimi). It addresses issue #2101 where generated SKILL.md files contained bare relative paths that don’t exist from the workspace root.

Changes:

Add extension-relative path rewriting during SKILL.md rendering and thread source_dir through skill rendering for primary commands and aliases.
Add unit tests covering rewriting across multiple subdirs, Kimi support, aliases, and “no subdirs” conservative behavior.
Add an opt-in integration test (env-var gated) that installs a real extension and validates end-to-end SKILL.md outputs.

Show a summary per file

File	Description
`src/specify_cli/agents.py`	Implements extension-relative path rewriting and applies it during SKILL.md rendering (with `source_dir` plumbing).
`tests/test_extensions.py`	Adds unit tests ensuring SKILL.md content rewrites extension-relative paths for Codex/Kimi and aliases.
`tests/test_integration_extension_skill_paths.py`	Adds env-gated integration tests to verify rewriting against a real installed extension.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 3/3 changed files
Comments generated: 0 new

mnriem · 2026-04-08T19:21:13Z

Just to make sure the non-rendered commands are installed in the .specify/extensions/ directory and then when the extension is enabled / disabled the SKILL.md files are put into the agent specific location using the skill rendering?

mnriem

Can you clarify if comment above is correct?

dhilipkumars · 2026-04-09T14:39:15Z

yup i have the same question is this specifically to support to codex ? my original PR did work for Gemini and AGY when i tested it.

#1840

mbachorik · 2026-04-10T10:05:59Z

@mnriem @dhilipkumars sorry for radio silence, I did not have time to come back to this. I hit several issues with recent change to spec-kit (originally I thought the fix is minimal to rewrite the relative paths). it has evolved into something bigger, I'll update the PR with proper explanation today or over the weekend.

Copilot

Pull request overview

Copilot reviewed 20 out of 21 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mnriem · 2026-04-10T13:20:11Z

@mbachorik Please address Copilot feedback

Copilot

Pull request overview

Copilot reviewed 22 out of 23 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 22 out of 23 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 22 out of 23 changed files in this pull request and generated 5 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…#2103 - Issue 1 (agents.py): Broaden Copilot behavior branch from `cmd_type == "agent"` to any non-empty behavior dict, so `execution: isolated` → `mode: agent` is injected and `behavior:`/`agents:` keys are stripped for all Copilot behavior values - Issue 2 (extensions.py): Treat non-dict/empty source `behavior:` as absent when merging manifest-level behavior, so `execution: agent` in the manifest correctly triggers the agent-deployment skip even when the source file has an invalid value - Issue 3 (integrations/base.py): Pass `agents:` escape-hatch overrides from template frontmatter through to `translate_behavior()` in `SkillsIntegration.setup()` - Issue 4 (agents.py): Narrow `_behavior_overridable` to only the two injected boolean defaults (`disable-model-invocation`, `user-invocable`); explicit source frontmatter values for `model`, `effort`, `context`, `agent`, `allowed-tools` now win - Issue 5 (extensions.py): Move `get_deployment_type` import to module scope and read each source file once per loop iteration instead of twice Tests: 26 new tests in tests/test_pr2103_reviews.py covering all paths for each issue Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 23 out of 24 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Pull request overview

Copilot reviewed 24 out of 25 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Files reviewed: 24/25 changed files
Comments generated: 3

Copilot

Pull request overview

Copilot reviewed 24 out of 25 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

…extension path rewriting

Copilot

Pull request overview

Copilot reviewed 24 out of 25 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot AI review requested due to automatic review settings April 6, 2026 09:50

mbachorik requested a review from mnriem as a code owner April 6, 2026 09:50

Copilot started reviewing on behalf of mbachorik April 6, 2026 09:51 View session

Copilot AI reviewed Apr 6, 2026

View reviewed changes

Comment thread src/specify_cli/agents.py Outdated

mbachorik marked this pull request as draft April 6, 2026 09:57

mnriem requested a review from Copilot April 7, 2026 12:28

Copilot started reviewing on behalf of mnriem April 7, 2026 12:29 View session

Copilot AI reviewed Apr 7, 2026

View reviewed changes

mnriem requested changes Apr 8, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings April 10, 2026 10:00

Copilot started reviewing on behalf of mbachorik April 10, 2026 10:00 View session

Copilot AI reviewed Apr 10, 2026

View reviewed changes

Comment thread src/specify_cli/extensions.py Outdated

Comment thread src/specify_cli/agents.py Outdated

Comment thread src/specify_cli/agents.py

Comment thread src/specify_cli/behavior.py

mbachorik changed the title ~~fix(extensions): rewrite extension-relative paths in generated SKILL.md files~~ feat(extensions): neutral behavior vocabulary with per-agent translation, fix LLM invocability Apr 11, 2026

Copilot AI review requested due to automatic review settings April 12, 2026 04:07

Copilot started reviewing on behalf of mbachorik April 12, 2026 04:08 View session

Copilot AI reviewed Apr 12, 2026

View reviewed changes

Comment thread src/specify_cli/agents.py Outdated

Comment thread src/specify_cli/integrations/base.py

Comment thread src/specify_cli/extensions.py Outdated

mbachorik force-pushed the fix/extension-skill-path-rewrite branch from 31dfd38 to 8852a23 Compare April 15, 2026 15:08

Copilot AI review requested due to automatic review settings April 15, 2026 15:08

Copilot started reviewing on behalf of mbachorik April 15, 2026 15:09 View session

Copilot AI reviewed Apr 15, 2026

View reviewed changes

Comment thread src/specify_cli/agents.py Outdated

Comment thread src/specify_cli/agents.py Outdated

Comment thread src/specify_cli/integrations/base.py Outdated

Comment thread src/specify_cli/extensions.py Outdated

mbachorik requested a review from Copilot April 19, 2026 17:51

Copilot started reviewing on behalf of mbachorik April 19, 2026 17:52 View session

Copilot AI reviewed Apr 19, 2026

View reviewed changes

Comment thread src/specify_cli/integrations/base.py Outdated

Comment thread src/specify_cli/agents.py Outdated

Comment thread src/specify_cli/agents.py Outdated

Comment thread src/specify_cli/extensions.py Outdated

Comment thread src/specify_cli/extensions.py Outdated

Copilot AI review requested due to automatic review settings April 28, 2026 08:17

Copilot started reviewing on behalf of mbachorik April 28, 2026 08:17 View session

Copilot AI reviewed Apr 28, 2026

View reviewed changes

Comment thread src/specify_cli/integrations/base.py

Comment thread src/specify_cli/extensions.py

Copilot AI review requested due to automatic review settings April 28, 2026 10:49

Copilot started reviewing on behalf of mbachorik April 28, 2026 10:49 View session

Copilot AI reviewed Apr 28, 2026

View reviewed changes

Comment thread src/specify_cli/integrations/base.py

mnriem requested a review from Copilot April 28, 2026 13:21

Copilot started reviewing on behalf of mnriem April 28, 2026 13:21 View session

Copilot AI reviewed Apr 28, 2026

View reviewed changes

Comment thread src/specify_cli/behavior.py Outdated

Comment thread src/specify_cli/integrations/base.py Outdated

Comment thread src/specify_cli/extensions.py

Copilot AI review requested due to automatic review settings April 29, 2026 19:41

Copilot started reviewing on behalf of mbachorik April 29, 2026 19:41 View session

Copilot AI reviewed Apr 29, 2026

View reviewed changes

Comment thread src/specify_cli/agents.py Outdated

Comment thread src/specify_cli/integrations/base.py

feat(extensions): neutral behavior vocabulary, invocability fix, and …

6bb1846

…extension path rewriting

mbachorik force-pushed the fix/extension-skill-path-rewrite branch from d34f068 to 6bb1846 Compare April 30, 2026 06:28

mbachorik requested a review from Copilot April 30, 2026 14:30

Copilot started reviewing on behalf of mbachorik April 30, 2026 14:31 View session

Copilot AI reviewed Apr 30, 2026

View reviewed changes

Comment thread tests/test_pr2103_reviews.py

Comment thread src/specify_cli/extensions.py

Conversation

mbachorik commented Apr 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Context

What

Why it is backwards compatible

Testing

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

mbachorik commented Apr 7, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Copilot's findings

Uh oh!

mnriem commented Apr 8, 2026

Uh oh!

mnriem left a comment

Choose a reason for hiding this comment

Uh oh!

dhilipkumars commented Apr 9, 2026

Uh oh!

mbachorik commented Apr 10, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mnriem commented Apr 10, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot's findings

Uh oh!

Uh oh!

mbachorik commented Apr 6, 2026 •

edited

Loading