collabSupport/services/voiceDiag
jmcqueen b802383441 Add Prisma SD-WAN voice-quality enrichment for /phonestatus + /voicediag
Introduces a full Palo Alto Prisma SD-WAN integration (dual-mode SASE
OAuth 2.0 / legacy CloudGenix auth, pagination, 429 backoff, session
priming) that surfaces per-path latency/jitter/loss/MOS, site
healthscore, link state, and alarm data for a store. Wired into the
/phonestatus WAN follow-up and eight new /voicediag WAN checks graded
against ITU-T G.114 / RFC 3550 defaults (env-overridable via
WAN_STANDARD_*).

Also adds a shape-aware detail renderer for /voicediag (per-link
tables with verdict icons instead of a stringified JSON dump) and a
--window flag (15m / 1h / 6h / 24h / 1d, env default via
WAN_STANDARD_WINDOW_MINUTES) so operators can widen the look-back
without redeploying. scripts/prismaProbe.js is bundled as a CLI for
schema iteration against a live tenant.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-09 09:45:29 -04:00
..
checks Add Prisma SD-WAN voice-quality enrichment for /phonestatus + /voicediag 2026-07-09 09:45:29 -04:00
README.md Add Prisma SD-WAN voice-quality enrichment for /phonestatus + /voicediag 2026-07-09 09:45:29 -04:00
voiceDiagService.js Add Prisma SD-WAN voice-quality enrichment for /phonestatus + /voicediag 2026-07-09 09:45:29 -04:00

/voicediag

Rules-engine style diagnostic for the store's Webex Calling per-user configuration. The orchestrator (voiceDiagService.js) walks a registry of check modules, gathers their CheckResult objects, and hands them to the renderer + adaptive-card layer in commands/voiceDiag.js.

Command

/voicediag <storeNum>                 default: hides OK checks, posts fixable cards
/voicediag <storeNum> detail          include OK checks + expand every details block
/voicediag <storeNum> --only dnd,callForwarding
/voicediag <storeNum> --window 24h    override the WAN look-back window (default 15m)
/voicediag list-checks                enumerate every registered check + its scope

HTTP path: GET /voicediag?storeNum=<n>[&detailed=true][&only=dnd,callWaiting][&window=24h].

--window accepts Nm / Nh / Nd shorthand (e.g. 15m, 1h, 6h, 24h, 1d) or a bare integer of minutes. Applies to the WAN healthscore + LQM (per-path latency/jitter/loss/MOS) fetches. Alarms are floored at 60m regardless — sub-hour alarm queries are usually too noisy to be actionable. Global default is WAN_STANDARD_WINDOW_MINUTES (env, defaults to 15). HTTP callers get the markdown snapshot only — remediation cards are chat-only.

Prerequisites

All checks call /v1/people/{personId}/features/{feature} (with a few Webex-namespace variants noted inline in each check module). The scopes required are:

  • spark-admin:people_read — every read (all checks)
  • spark-admin:people_write — remediations (disable DND, clear forwarding, enable call waiting, disable call intercept)

These are the same scopes /webexhost and /offboarduser already use, so no operator work is needed to enable /voicediag on an existing deployment. If the token ever loses those scopes, each check self-reports as skipped: scope missing — the run still completes, and the phone-online summary keeps working.

About the URL scheme

Early drafts of this feature documented /v1/telephony/config/people/{id}/callSettings/*. That path family returns 404 "no static resource" from the Webex API gateway — it is not a live surface. The correct admin path is /v1/people/{id}/features/{feature}, per Webex's User Call Settings docs. A couple of features use different feature-name segments than their check id — most notably intercept (not callIntercept); see each check module for the exact endpoint. The runner distinguishes routing-404s ("URL moved, update the check") from "not applicable" 404s ("this person isn't a calling user") using the no static resource marker in the response body.

Store voice standards

The store phone standard is enforced by the checks below. Every check descriptor exports a standards object so the desired state is legible from the check file without reading run(). Changes here are meant to be a two-step change: update standards, then teach run() to interpret it — the checks compare the live state against standards and emit the severity in the "when non-compliant" column.

Per-user Webex Calling standards

Check Standard Non-compliant severity Auto-remediation
dnd enabled: false warn disable_dnd
callForwarding {always, busy, noAnswer}.enabled: false error clear_call_forwarding
callWaiting enabled: true warn enable_call_waiting
callIntercept enabled: false error disable_call_intercept
voicemail enabled: true, all three send*Calls.enabled: false, mwiEnabled: true error on send-to-VM, warn on MWI-off / disabled / off-org email stop_sending_to_voicemail for the send-to-VM path only
hoteling enabled: false warn disable_hoteling
executiveAssistant type: 'UNASSIGNED' warn none — Control Hub cleanup
outgoingPermission no high-impact call type BLOCKED (LOCAL, NATIONAL, TOLL_FREE, TOLL, INTERNATIONAL) warn none — location-scoped fix

callForwarding is intentionally at error severity: forwarding active on a store line silently drops customer calls, and it's the single most common voice ticket. Voicemail's error path is narrower — only the three send-* triggers upgrade to error, since that also silently swallows calls. MWI-off, off-org email forwarding, and voicemail-disabled all stay at warn.

Switchport / port-hygiene standards

These reuse the phone-status snapshot from /phonestatus — no extra Webex API calls — and cross-reference against the Meraki port config that already flows through services/enrichment/merakiEnrichment.js. Trunk uplinks are treated as a visibility boundary: the phone is behind a non-Meraki switch (typically a Cisco stack in a store) and per-port policy isn't ours to enforce, so downstream VLAN / PoE / admin-state checks defer to the operator.

Check Standard Non-compliant severity Notes
portType portType: 'access' on every wired phone / DECT base warn Trunk uplinks flagged so operator checks the downstream switch
portVlan vlan === VOICE_STANDARD_PHONE_VLAN (default 102) warn Env-configurable — VLAN may move from data-side to a proper voice VLAN in the future
portPoe poeEnabled: true warn Skipped for trunk-uplinked devices
portEnabled portEnabled: true error Admin-disabled port → phone is dead

No auto-remediation on any port check — Meraki port-config PUTs are a separate scope of work; the operator handles fixes in the Meraki Dashboard.

Environment overrides

Both port-hygiene knobs live in .env:

Var Default Purpose
VOICE_STANDARD_PHONE_VLAN 102 Expected VLAN for a store phone. Set per-site if the fleet moves onto a proper voice VLAN.
VOICE_STANDARD_ENABLED true Global kill-switch for the port-hygiene bucket. false silences portType / portVlan / portPoe / portEnabled while Meraki cleanup is in progress. Feature-config checks always run.

WAN standards (Prisma SD-WAN)

Sources Prisma SD-WAN metrics (see integrations/paloalto/) for the store's site and grades per-path LQM + healthscore + alarms against ITU-T G.114 / RFC 3550 references. Diagnostic-only — no auto-remediation, since WAN config PUTs are the operator's job in the Prisma portal, not the bot's. All checks skip cleanly with "not applicable — missing sdwanSite" for non-Prisma-managed sites, so the WAN bucket adds no noise to stores that live entirely behind Meraki.

Data flow: voiceDiagService.buildContext() calls collectSdwanForStore(storeNum) in parallel with the Webex + Meraki fetches. That composer resolves the store to a Prisma site (CG${pad5(storeNum)} — e.g. store 782 → CG00782), pulls elements + healthscore + LinkState + 4 LQM metrics + recent alarms in parallel, and normalises to a stable shape the checks + the /phonestatus follow-up renderer share.

Check Standard Non-compliant severity Notes
wanSite Prisma site resolves for the store skipped when unresolved Info-only anchor — surfaces site + element + link counts
wanHealthscore >= WAN_STANDARD_HEALTHSCORE_WARN (default 80) warn < 80, error < 60 Prisma AIOps composite (0-100)
wanLinkState Every WAN path up error if any path down No remediation — physical / carrier work
wanLatency <= WAN_STANDARD_LATENCY_WARN_MS (default 150ms) warn > 150ms, error > 400ms ITU-T G.114 one-way reference. Worst path drives severity.
wanJitter <= WAN_STANDARD_JITTER_WARN_MS (default 30ms) warn > 30ms, error > 50ms RFC 3550 jitter-buffer envelope
wanLoss <= WAN_STANDARD_LOSS_WARN_PCT (default 1%) warn > 1%, error > 3% G.711 PLC tolerance
wanMos >= WAN_STANDARD_MOS_WARN (default 4.0) warn < 4.0, error < 3.5 ITU-T P.800 MOS scale. LOW is bad — the accessor flips the comparison.
wanAlarms Zero critical + zero major alarms in the last hour warn on major, error on critical Minor alarms are info-only. Pass-through of Prisma severity.

Environment overrides (all optional — defaults match the standards above):

Var Default Purpose
WAN_STANDARD_LATENCY_WARN_MS 150 Latency warn threshold, ms
WAN_STANDARD_LATENCY_ERROR_MS 400 Latency error threshold, ms
WAN_STANDARD_JITTER_WARN_MS 30 Jitter warn threshold, ms
WAN_STANDARD_JITTER_ERROR_MS 50 Jitter error threshold, ms
WAN_STANDARD_LOSS_WARN_PCT 1 Packet-loss warn threshold, %
WAN_STANDARD_LOSS_ERROR_PCT 3 Packet-loss error threshold, %
WAN_STANDARD_MOS_WARN 4.0 MOS warn threshold (LOW is bad)
WAN_STANDARD_MOS_ERROR 3.5 MOS error threshold
WAN_STANDARD_HEALTHSCORE_WARN 80 Site healthscore warn threshold (0-100)
WAN_STANDARD_HEALTHSCORE_ERROR 60 Site healthscore error threshold
WAN_STANDARD_ENABLED true Global kill-switch for the WAN bucket. false silences all 8 WAN checks while a Prisma cleanup / integration validation is in flight.

Prisma credentials and auth-mode selection live under the Palo Alto Prisma SD-WAN block in .env.example (SASE OAuth 2.0 recommended, legacy CloudGenix session-token supported for backward compatibility). See that block for the full env inventory.

Debugging Prisma schema drift

Prisma SD-WAN's monitor API schemas vary between tenants — request body shapes documented on pan.dev or in LIVEcommunity examples frequently fail schema check on this tenant with cryptic 400 messages like "$.filter.wan_interfaces: is not defined in the schema" or "$.view: does not have a value in the enumeration [summary, timeseries]". Iterating on those in-band via /phonestatus is painful (full DECT + Meraki + WAN pipeline runs every attempt).

Use the standalone probe script instead:

node scripts/prismaProbe.js discover 782
node scripts/prismaProbe.js try-shapes health <siteId>
node scripts/prismaProbe.js try-shapes lqm <siteId> <wiId1,wiId2>
node scripts/prismaProbe.js raw POST /sdwan/v3.7/api/events/query --body '{"limit":{"count":5}}'

try-shapes fires a curated list of candidate request bodies and prints which pass schema check (with response-shape summary) and which fail (with the exact SCHEMA_CHECK_FAIL field). When you find a winning shape, port it into integrations/paloalto/metrics.js and add a regression test in tests/paloalto.metrics.test.js so the tenant's quirk doesn't get "helpfully cleaned up" by a future refactor. Same auth stack as the bot, no side effects.

Apply-all-N-fixes card

When two or more checks return fixable results, /voicediag posts one extra adaptive card at the bottom offering to apply the whole batch in a single click. The individual per-issue cards stay on-screen so operators can still pick and choose; the combined card is a shortcut for the common "everything looks right, do it all" case. The batch executes each fix in sequence (not parallel) so audit lines stay readable and per-person Webex API write throttling doesn't stack; failures accumulate into a final summary line rather than aborting the run.

Dispatcher-side, the combined card uses the same pendingVoiceFixes map and the same voicediag branch in index.js — the payload's combined: true flag + the confirm_voicediag_all / cancel_voicediag_all action ids are what pick the batch handler over the single-fix handler.

Adding a new check

  1. Create services/voiceDiag/checks/<newCheck>.js and export a descriptor:

    // The standards block is *the* source of truth for the desired
    // state. Keep the run() comparison in sync — regression tests
    // in tests/voiceDiag.checks.test.js assert every check has one.
    export const MY_NEW_STANDARDS = Object.freeze({ enabled: false });
    
    export const myNewCheck = {
      id: 'myNew',
      label: 'My New Check',
      requires: ['personId'],          // subset of ['personId','phoneStatus','telephonyProfile']
      scope: 'spark-admin:people_read',
      standards: MY_NEW_STANDARDS,
      async run(ctx) {
        const data = await ctx.webex.request('GET', `people/${ctx.personId}/features/whatever`);
        if (!!data?.enabled === MY_NEW_STANDARDS.enabled) {
          return { status: 'ok', message: 'Compliant.', details: null, remediation: null };
        }
        return {
          status: 'warn',              // 'ok' | 'warn' | 'error' | 'skipped'
          message: 'Human sentence for the chat row.',
          details: { relevant: 'facts' },
          remediation: {               // optional
            action: 'fix_my_thing',    // globally unique remediation id
            title: 'Fix My Thing',
            summary: 'One-line description of what the fix will do.',
            payload: { personId: ctx.personId, personLabel: ctx.personLabel, storeNum: ctx.storeNum },
          },
        };
      },
      remediations: {                 // optional
        async fix_my_thing(bot, data, requester) {
          // do the PUT, audit, reply
        },
      },
    };
    
  2. Register it in services/voiceDiag/checks/index.js at the position you want the renderer to output it.

  3. Add a describe-level block to tests/voiceDiag.checks.test.js covering the enabled / disabled / 403 branches.

That's it — no changes to the runner, renderer, command handler, index.js dispatcher, or commands/registry.js. The single confirm_voicediag / cancel_voicediag action namespace routes through the shared dispatcher, which looks the remediation up in the central registry built at module load. Remediation id collisions throw at import so you'll notice immediately.

CheckResult shape

type Severity = 'ok' | 'warn' | 'error' | 'skipped';

type Remediation = {
  action: string;             // globally unique across all checks
  title: string;              // button label
  summary?: string;           // optional card body line
  payload: Record<string, unknown>;  // handed to remediations[action](bot, payload, requester)
};

type CheckResult = {
  id: string;                 // set from check.id automatically
  label: string;              // set from check.label automatically
  status: Severity;
  message: string;
  details?: object | null;    // shown under `detail` mode
  remediation?: Remediation | null;
};

Error handling

Never throw from run(). The runner wraps every check in a try/catch and converts:

  • HTTP 401/403/404 → status: 'skipped' with a "missing scope" hint pointing at check.scope.
  • Any other error → status: 'error' with the API message.

So if you can't reason about a specific 5xx, just let it bubble — the runner will surface it.