collabSupport/services/voiceDiag

/voicediag

Rules-engine style diagnostic for the store's Webex Calling per-user configuration. The orchestrator (voiceDiagService.js) walks a registry of check modules, gathers their CheckResult objects, and hands them to the renderer + adaptive-card layer in commands/voiceDiag.js.

Command

/voicediag <storeNum>                 default: hides OK checks, posts fixable cards
/voicediag <storeNum> detail          include OK checks + expand every details block
/voicediag <storeNum> --only dnd,callForwarding
/voicediag <storeNum> --window 24h    narrow the WAN look-back (default 7d)
/voicediag list-checks                enumerate every registered check + its scope

HTTP path: GET /voicediag?storeNum=<n>[&detailed=true][&only=dnd,callWaiting][&window=15m].

--window accepts Nm / Nh / Nd shorthand (e.g. 15m, 1h, 6h, 24h, 1d, 7d) or a bare integer of minutes; hard-capped at 7d. Applies to the WAN healthscore + LQM (per-path latency/jitter/loss/MOS)

• per-app voice-DPI fetches + alarms. Alarms are floored at 60m regardless — sub-hour alarm queries are usually too noisy to be actionable. Global default is WAN_STANDARD_WINDOW_MINUTES (env, defaults to 10080 / 7 days). The 7d default was chosen because per-app DPI metrics (Webex_Calling_RTP etc.) only get datapoints when calls actually happen — sporadic stores need a wider window for worst-window statistics to be meaningful. For live-incident triage where you want a fresh snapshot, pass --window 1h or --window 24h. The same default is used by the /phonestatus WAN follow-up (no CLI override on that surface — set the env if you want a different value globally). HTTP callers get the markdown snapshot only — remediation cards are chat-only.

Voice path attribution (WAN tunnels / which pipe)

See VOICE_PATH_ATTRIBUTION.md for Phase 1 surfaces (runtime circuit status, overlay tunnels, DPI by path_type) and Phase 2 discovery status for per-circuit / per-handset attribution.

Enable optional per-waninterface app DPI fan-out with:

PRISMA_VOICE_PATH_ATTRIBUTION=1

Prerequisites

All checks call /v1/people/{personId}/features/{feature} (with a few Webex-namespace variants noted inline in each check module). The scopes required are:

• spark-admin:people_read — every read (all checks)
• spark-admin:people_write — remediations (disable DND, clear forwarding, enable call waiting, disable call intercept)

These are the same scopes /webexhost and /offboarduser already use, so no operator work is needed to enable /voicediag on an existing deployment. If the token ever loses those scopes, each check self-reports as skipped: scope missing — the run still completes, and the phone-online summary keeps working.

About the URL scheme

Early drafts of this feature documented /v1/telephony/config/people/{id}/callSettings/*. That path family returns 404 "no static resource" from the Webex API gateway — it is not a live surface. The correct admin path is /v1/people/{id}/features/{feature}, per Webex's User Call Settings docs. A couple of features use different feature-name segments than their check id — most notably intercept (not callIntercept); see each check module for the exact endpoint. The runner distinguishes routing-404s ("URL moved, update the check") from "not applicable" 404s ("this person isn't a calling user") using the no static resource marker in the response body.

Store voice standards

The store phone standard is enforced by the checks below. Every check descriptor exports a standards object so the desired state is legible from the check file without reading run(). Changes here are meant to be a two-step change: update standards, then teach run() to interpret it — the checks compare the live state against standards and emit the severity in the "when non-compliant" column.

Per-user Webex Calling standards

Check	Standard	Non-compliant severity	Auto-remediation
dnd	enabled: false	warn	disable_dnd
callForwarding	{always, busy, noAnswer}.enabled: false	error	clear_call_forwarding
callWaiting	enabled: true	warn	enable_call_waiting
callIntercept	enabled: false	error	disable_call_intercept
voicemail	enabled: true, all three send*Calls.enabled: false, mwiEnabled: true	error on send-to-VM, warn on MWI-off / disabled / off-org email	stop_sending_to_voicemail for the send-to-VM path only
hoteling	enabled: false	warn	disable_hoteling
executiveAssistant	type: 'UNASSIGNED'	warn	none — Control Hub cleanup
outgoingPermission	no high-impact call type BLOCKED (LOCAL, NATIONAL, TOLL_FREE, TOLL, INTERNATIONAL)	warn	none — location-scoped fix

callForwarding is intentionally at error severity: forwarding active on a store line silently drops customer calls, and it's the single most common voice ticket. Voicemail's error path is narrower — only the three send-* triggers upgrade to error, since that also silently swallows calls. MWI-off, off-org email forwarding, and voicemail-disabled all stay at warn.

Switchport / port-hygiene standards

These reuse the phone-status snapshot from /phonestatus — no extra Webex API calls — and cross-reference against the Meraki port config that already flows through services/enrichment/merakiEnrichment.js. Trunk uplinks are treated as a visibility boundary: the phone is behind a non-Meraki switch (typically a Cisco stack in a store) and per-port policy isn't ours to enforce, so downstream VLAN / PoE / admin-state checks defer to the operator.

Check	Standard	Non-compliant severity	Notes
portType	portType: 'access' on every wired phone / DECT base	warn	Trunk uplinks flagged so operator checks the downstream switch
portVlan	vlan === VOICE_STANDARD_PHONE_VLAN (default 102)	warn	Env-configurable — VLAN may move from data-side to a proper voice VLAN in the future
portPoe	poeEnabled: true	warn	Skipped for trunk-uplinked devices
portEnabled	portEnabled: true	error	Admin-disabled port → phone is dead

No auto-remediation on any port check — Meraki port-config PUTs are a separate scope of work; the operator handles fixes in the Meraki Dashboard.

Environment overrides

Both port-hygiene knobs live in .env:

Var	Default	Purpose
VOICE_STANDARD_PHONE_VLAN	102	Expected VLAN for a store phone. Set per-site if the fleet moves onto a proper voice VLAN.
VOICE_STANDARD_ENABLED	true	Global kill-switch for the port-hygiene bucket. false silences portType / portVlan / portPoe / portEnabled while Meraki cleanup is in progress. Feature-config checks always run.

WAN standards (Prisma SD-WAN)

Sources Prisma SD-WAN metrics (see integrations/paloalto/) for the store's site and grades per-path LQM + healthscore + alarms against ITU-T G.114 / RFC 3550 references. Diagnostic-only — no auto-remediation, since WAN config PUTs are the operator's job in the Prisma portal, not the bot's. All checks skip cleanly with "not applicable — missing sdwanSite" for non-Prisma-managed sites, so the WAN bucket adds no noise to stores that live entirely behind Meraki.

Data flow: voiceDiagService.buildContext() calls collectSdwanForStore(storeNum) in parallel with the Webex + Meraki fetches. That composer resolves the store to a Prisma site (CG${pad5(storeNum)} — e.g. store 782 → CG00782), pulls elements + healthscore + LinkState + 4 LQM metrics + recent alarms in parallel, and normalises to a stable shape the checks + the /phonestatus follow-up renderer share.

Check	Standard	Non-compliant severity	Notes
wanSite	Prisma site resolves for the store	skipped when unresolved	Info-only anchor — surfaces site + element + link counts
wanHealthscore	>= WAN_STANDARD_HEALTHSCORE_WARN (default 80)	warn < 80, error < 60	Prisma AIOps composite (0-100)
wanLinkState	Every WAN path up	error if any path down	No remediation — physical / carrier work
wanLatency	<= WAN_STANDARD_LATENCY_WARN_MS (default 150ms)	warn > 150ms, error > 400ms	ITU-T G.114 one-way reference. Worst path drives severity.
wanJitter	<= WAN_STANDARD_JITTER_WARN_MS (default 30ms)	warn > 30ms, error > 50ms	RFC 3550 jitter-buffer envelope
wanLoss	<= WAN_STANDARD_LOSS_WARN_PCT (default 1%)	warn > 1%, error > 3%	G.711 PLC tolerance
wanMos	>= WAN_STANDARD_MOS_WARN (default 4.0)	warn < 4.0, error < 3.5	ITU-T P.800 MOS scale. LOW is bad — the accessor flips the comparison.
wanAppRtpMos	Worst 5-min window >= WAN_STANDARD_APP_MOS_WARN (default 4.0)	warn < 4.0, error < 3.5	Real DPI measurement on the configured voice app's RTP frames (e.g. Webex_Calling_RTP, rtp-base). Graded against the WORST window in the series — catches transient degradation link-probe averages hide. Skipped when PRISMA_APP_ID_VOICE unset.
wanAppRtpLoss	Worst 5-min window <= WAN_STANDARD_APP_LOSS_WARN_PCT (default 5%)	warn > 5%, error > 15%	DPI packet-loss for the same voice app. Same feature gate.
wanAppRtpJitter	Worst 5-min window <= WAN_STANDARD_APP_JITTER_WARN_MS (default 30ms)	warn > 30ms, error > 50ms	DPI jitter for the same voice app. Same feature gate.
wanAlarms	Zero critical + zero major alarms in the last hour	warn on major, error on critical	Minor alarms are info-only. Pass-through of Prisma severity.
wanTunnels	All overlay / VPN tunnels up	error if any down; warn if recent alarms	Distinguishes overlay-down vs physical-up.

Environment overrides (all optional — defaults match the standards above):

Var	Default	Purpose
WAN_STANDARD_LATENCY_WARN_MS	150	Latency warn threshold, ms
WAN_STANDARD_LATENCY_ERROR_MS	400	Latency error threshold, ms
WAN_STANDARD_JITTER_WARN_MS	30	Jitter warn threshold, ms
WAN_STANDARD_JITTER_ERROR_MS	50	Jitter error threshold, ms
WAN_STANDARD_LOSS_WARN_PCT	1	Packet-loss warn threshold, %
WAN_STANDARD_LOSS_ERROR_PCT	3	Packet-loss error threshold, %
WAN_STANDARD_MOS_WARN	4.0	MOS warn threshold (LOW is bad)
WAN_STANDARD_MOS_ERROR	3.5	MOS error threshold
WAN_STANDARD_HEALTHSCORE_WARN	80	Site healthscore warn threshold (0-100)
WAN_STANDARD_HEALTHSCORE_ERROR	60	Site healthscore error threshold
WAN_STANDARD_APP_MOS_WARN	4.0	Per-app worst-window MOS warn threshold
WAN_STANDARD_APP_MOS_ERROR	3.5	Per-app worst-window MOS error threshold
WAN_STANDARD_APP_LOSS_WARN_PCT	5	Per-app worst-window packet-loss warn threshold, %
WAN_STANDARD_APP_LOSS_ERROR_PCT	15	Per-app worst-window packet-loss error threshold, %
WAN_STANDARD_APP_JITTER_WARN_MS	30	Per-app worst-window jitter warn threshold, ms
WAN_STANDARD_APP_JITTER_ERROR_MS	50	Per-app worst-window jitter error threshold, ms
PRISMA_APP_ID_VOICE	(unset)	Numeric Prisma app id for the tenant's voice app. Recommended: Webex_Calling_RTP for Webex Calling shops (excludes non-Webex UDP noise), otherwise rtp-base for the generic RTP signature. When set, /phonestatus + /voicediag pull real DPI voice-quality metrics from Prisma's Application Path Details endpoint. Discover with npm run prisma:probe -- appdefs webex (or rtp) or from the Prisma UI URL. Leave unset to skip the extra API calls.
PRISMA_APP_NAME_VOICE	voice	Display label for the app configured above. Set to the exact Prisma UI label (e.g. Webex_Calling_RTP) so section headers and check messages cross-reference cleanly.
PRISMA_APP_ID_RTP_BASE	(unset)	Deprecated. Backwards-compat fallback for the original release. Still honored if set and PRISMA_APP_ID_VOICE is unset (with a one-time deprecation warning in the log); display name defaults to rtp-base in that case. Rename to PRISMA_APP_ID_VOICE at your convenience.
WAN_STANDARD_ENABLED	true	Global kill-switch for the WAN bucket. false silences all 11 WAN checks while a Prisma cleanup / integration validation is in flight.

Prisma credentials and auth-mode selection live under the Palo Alto Prisma SD-WAN block in .env.example (SASE OAuth 2.0 recommended, legacy CloudGenix session-token supported for backward compatibility). See that block for the full env inventory.

Debugging Prisma schema drift

Prisma SD-WAN's monitor API schemas vary between tenants — request body shapes documented on pan.dev or in LIVEcommunity examples frequently fail schema check on this tenant with cryptic 400 messages like "$.filter.wan_interfaces: is not defined in the schema" or "$.view: does not have a value in the enumeration [summary, timeseries]". Iterating on those in-band via /phonestatus is painful (full DECT + Meraki + WAN pipeline runs every attempt).

Use the standalone probe script instead:

node scripts/prismaProbe.js discover 782
node scripts/prismaProbe.js try-shapes health <siteId>
node scripts/prismaProbe.js try-shapes lqm <siteId> <wiId1,wiId2>
node scripts/prismaProbe.js raw POST /sdwan/v3.7/api/events/query --body '{"limit":{"count":5}}'

try-shapes fires a curated list of candidate request bodies and prints which pass schema check (with response-shape summary) and which fail (with the exact SCHEMA_CHECK_FAIL field). When you find a winning shape, port it into integrations/paloalto/metrics.js and add a regression test in tests/paloalto.metrics.test.js so the tenant's quirk doesn't get "helpfully cleaned up" by a future refactor. Same auth stack as the bot, no side effects.

Apply-all-N-fixes card

When two or more checks return fixable results, /voicediag posts one extra adaptive card at the bottom offering to apply the whole batch in a single click. The individual per-issue cards stay on-screen so operators can still pick and choose; the combined card is a shortcut for the common "everything looks right, do it all" case. The batch executes each fix in sequence (not parallel) so audit lines stay readable and per-person Webex API write throttling doesn't stack; failures accumulate into a final summary line rather than aborting the run.

Dispatcher-side, the combined card uses the same pendingVoiceFixes map and the same voicediag branch in index.js — the payload's combined: true flag + the confirm_voicediag_all / cancel_voicediag_all action ids are what pick the batch handler over the single-fix handler.

Adding a new check

1. Create services/voiceDiag/checks/<newCheck>.js and export a descriptor:

   // The standards block is *the* source of truth for the desired
   // state. Keep the run() comparison in sync — regression tests
   // in tests/voiceDiag.checks.test.js assert every check has one.
   export const MY_NEW_STANDARDS = Object.freeze({ enabled: false });
   
   export const myNewCheck = {
     id: 'myNew',
     label: 'My New Check',
     requires: ['personId'],          // subset of ['personId','phoneStatus','telephonyProfile']
     scope: 'spark-admin:people_read',
     standards: MY_NEW_STANDARDS,
     async run(ctx) {
       const data = await ctx.webex.request('GET', `people/${ctx.personId}/features/whatever`);
       if (!!data?.enabled === MY_NEW_STANDARDS.enabled) {
         return { status: 'ok', message: 'Compliant.', details: null, remediation: null };
       }
       return {
         status: 'warn',              // 'ok' | 'warn' | 'error' | 'skipped'
         message: 'Human sentence for the chat row.',
         details: { relevant: 'facts' },
         remediation: {               // optional
           action: 'fix_my_thing',    // globally unique remediation id
           title: 'Fix My Thing',
           summary: 'One-line description of what the fix will do.',
           payload: { personId: ctx.personId, personLabel: ctx.personLabel, storeNum: ctx.storeNum },
         },
       };
     },
     remediations: {                 // optional
       async fix_my_thing(bot, data, requester) {
         // do the PUT, audit, reply
       },
     },
   };
   

2. Register it in services/voiceDiag/checks/index.js at the position you want the renderer to output it.

3. Add a describe-level block to tests/voiceDiag.checks.test.js covering the enabled / disabled / 403 branches.

That's it — no changes to the runner, renderer, command handler, index.js dispatcher, or commands/registry.js. The single confirm_voicediag / cancel_voicediag action namespace routes through the shared dispatcher, which looks the remediation up in the central registry built at module load. Remediation id collisions throw at import so you'll notice immediately.

CheckResult shape

type Severity = 'ok' | 'warn' | 'error' | 'skipped';

type Remediation = {
  action: string;             // globally unique across all checks
  title: string;              // button label
  summary?: string;           // optional card body line
  payload: Record<string, unknown>;  // handed to remediations[action](bot, payload, requester)
};

type CheckResult = {
  id: string;                 // set from check.id automatically
  label: string;              // set from check.label automatically
  status: Severity;
  message: string;
  details?: object | null;    // shown under `detail` mode
  remediation?: Remediation | null;
};

Error handling

Never throw from run(). The runner wraps every check in a try/catch and converts:

• HTTP 401/403/404 → status: 'skipped' with a "missing scope" hint pointing at check.scope.
• Any other error → status: 'error' with the API message.

So if you can't reason about a specific 5xx, just let it bubble — the runner will surface it.