Testing Strategy

DSH's testing policy (docs/testing.md) requires assembled coverage for anything model- or UI-visible. Every tool here is model-visible, so every tool needs tests.

Layers

1. Unit — client

tests/client.test.ts

Case	Assertion
Token bucket allows `burst` immediately	N+1th request waits
`error 6` retried with backoff	call count = `maxAttempts`, delays monotonic
`error 5` not retried	one attempt, typed `VK_AUTH_FAILED`
`error 14` returns captcha sid	`VK_CAPTCHA_REQUIRED.captchaSid` set
`error 100` includes param name	message names the offending param
`exec.signal` aborted mid-request	rejects with an abort error, no retry
`execute` batching packs ≤25	one HTTP request for 25 sub-calls
Response `{error}` vs `{response}`	both unwrapped correctly

Use nock or msw to intercept. No real network in unit tests.

2. Contract — every tool

tests/tools.test.ts, table-driven over all 30 tools.

For each tool assert:

Schema validity — output.schema accepts a representative canonical value and rejects a malformed one.
Arg validation — defineTool rejects a missing required arg and accepts an optional one.
Canonical output — execute returns a value matching the schema, not prose.
render totality — output.render(args, value) returns ContentBlock[] for boundary values: empty arrays, zero counts, null optionals, very long text.
Presenter purity — calling presentCall(args) twice returns structurally identical results; no I/O is attempted (assert by running with a fetch stub that throws).
Presenter fallback — malformed args → undefined (generic card), never a throw.

Table shape:

const cases: Array<{
  tool: string
  validArgs: unknown
  canonicalValue: unknown
  invalidArgs: unknown
}> = [ ... ]

3. Integration — against a mock VK server

tests/integration.test.ts boots a local HTTP server that speaks the VK protocol ({response} / {error}, execute batching).

Scenario	Steps
Post lifecycle	`wall_post` → `wall_edit` → `wall_pin` → `wall_delete`, assert request sequence and canonical outputs
Batch ban	`ban_manage` with 500 ids → assert 20 `execute` batches of 25, per-id results, partial failures surfaced
Media upload	`media_upload` with a fake file → assert 3-step sequence, single-use URL, retry restarts from step 1
LongPoll recovery	server returns `failed:1` then `failed:2` then normal → assert `ts` recovery, key refresh, no duplicate delivery
LongPoll abort	stop job mid-poll → assert the in-flight request is aborted and the loop exits
Scope missing	community token only, call `members_list` → `VK_SCOPE_MISSING` naming `groups`
Tier deny	`admin_level: 1`, call `community_update` → denied by guard, no HTTP request issued

4. Moderation engine

tests/moderation.test.ts

Case	Assertion
Keyword match, Cyrillic	matches case-insensitively
Regex invalid	rule rejected at load with a clear error, other rules still load
Exempt staff	admin-level user never acted on
Warn threshold	3 warnings → ban, counter reset
First-match-wins	two matching rules → only the first action executes
Outbound gate	agent post matching a `delete` rule is blocked before dispatch
Dry-run	rules evaluate and audit but no VK write happens

5. Policy layer

tests/policy.test.ts

Case	Assertion
Destructive + `requireConfirmForDestructive`	pre-execute returns `ask`
Destructive + flag off	returns `allow`
Guard denies below tier	guard returns a reason; a later `allow` listener cannot undo it
`raw_api` disabled	denied with a message pointing at `safety.rawApiEnabled`
`raw_api` enabled, method not allowlisted	denied
Audit on every mutating call	`tools/result` appended exactly one record

6. Replay purity

tests/replay.test.ts

Simulate the session-log replay path: call presentCall and presentResult with logged args and a persisted result.meta, in a process where fetch throws. Both must return the same card they returned live. This is the test that catches the most common DSH plugin bug.

Coverage targets

Area	Target
`vk/client.ts`	95%
`vk/errors.ts`	100%
`vk/methods.ts`	90%
`tools/*`	85% (schema + one happy path + one error path each)
`policy/*`	100% (it is small and it is the safety layer)
`events/*`	85%
`moderation/*`	90%

No global threshold below these — a global number hides an untested safety module behind a well-tested HTTP client.

What is not tested automatically

Real VK API behaviour (rate limits, captcha, flood control). Verified manually against a test community once per release.
UI cards in a real Web client. Verified manually with a screenshot per card kind.
Callback API end-to-end (needs a public endpoint).

Test community

Integration tests that must hit the real API use a dedicated test community with a community token stored in the CI environment. They are marked @live and excluded from the default npm test run; npm run test:live runs them.

Commands

npm test              # unit + contract + integration (mocked) + policy + replay
npm run test:watch    # vitest in watch mode
npm run test:live     # @live tests against a real test community
npm run test:cov      # coverage report, fails below the per-area targets

Testing Strategy

Layers

1. Unit — client

2. Contract — every tool

3. Integration — against a mock VK server

4. Moderation engine

5. Policy layer

6. Replay purity

Coverage targets

What is not tested automatically

Test community

Commands

dsh-vk-group

Overview

Platform reference

Operating

Building

Reference