jmcqueen
c2e98d105e
ServiceChannel webhook processor, proposal approval cards, attachment auto-post, CollabSupport commands, and Docker deployment configuration. Co-authored-by: Cursor <cursoragent@cursor.com>
264 lines
No EOL
14 KiB
Markdown
264 lines
No EOL
14 KiB
Markdown
# ServChan
|
||
|
||
Service Channel + Webex Bot for retail facilities coordination.
|
||
|
||
ServChan automatically creates dedicated Webex collaboration spaces for every ServiceChannel work order, posts rich updates (including AI-summarized descriptions), and provides useful slash commands for technicians and support teams.
|
||
|
||
The goal of this document is to help you understand how the system actually works — both architecturally (how data and actions flow) and operationally (how to run, monitor, and debug it).
|
||
|
||
## How the System Works (High-Level Flow)
|
||
|
||
### 1. Webhook Path (ServiceChannel → Webex Spaces)
|
||
|
||
This is the core automated behavior:
|
||
|
||
1. ServiceChannel sends a webhook to `POST /webhook` (handled in `src/server/app.js`).
|
||
2. The route immediately calls `webhookProcessor.processWebhook(payload)`.
|
||
3. `webhookProcessor` (src/services/webhookProcessor.js) does the following:
|
||
- Uses a **per-workOrderId Mutex + queue** to safely handle multiple events for the same work order arriving at the same time.
|
||
- Checks the SQLite DB (`src/db/mappings.js`) to see if a Webex room already exists for this work order.
|
||
- If no room exists:
|
||
- Calls `webexService.createWorkOrderRoom(...)` to create a new group space under the configured team.
|
||
- Adds the default list of members.
|
||
- Stores the `workOrderId ↔ roomId` mapping in the database.
|
||
- Builds a message based on the `EventType` (`WorkOrderCreated`, `WorkOrderNoteAdded`, etc.).
|
||
- For `WorkOrderCreated`, it calls `summarizeTicketDescription()` from `src/integrations/xai/client.js` to turn the messy ServiceChannel description into something readable.
|
||
- Posts the message using `webexService.sendMarkdown(...)`.
|
||
|
||
4. `webexService` (src/services/webexService.js) is a thin wrapper. It currently delegates to `botClient` but gives us a place to add ServChan-specific logic or change the underlying client later without touching the processor.
|
||
|
||
### 2. Command Path (User → Bot)
|
||
|
||
Users interact with the bot via mentions in Webex:
|
||
|
||
- The Webex Framework is initialized in `src/bot/index.js`.
|
||
- All commands are routed through a single `Framework.hears(...)` handler.
|
||
- Commands are dispatched to handlers in `src/commands/` (`help.js`, `avStatus.js`, `woSummary.js`, etc.).
|
||
- Many of these handlers call the external CollabSupport service (configured via `CS_API_BASE`) rather than doing heavy work locally.
|
||
|
||
### Key Classes / Modules and Their Responsibilities
|
||
|
||
| Module / Class | Role |
|
||
|-----------------------------------|------|
|
||
| `index.js` | Thin bootstrap. Creates DB connection, instantiates `webhookProcessor` + `webexService`, starts the bot and Express server. |
|
||
| `webhookProcessor.js` | The heart of automation. Owns concurrency control (mutex/queue), room lifecycle decisions, and message construction. |
|
||
| `webexService.js` | Thin service layer on top of the Webex client. Preferred interface for anything that needs to talk to Webex. |
|
||
| `botClient.js` (in `integrations/webex/`) | Low-level Webex bot operations using the bot token (create rooms, post messages, add members). |
|
||
| `adminClient.js` (in `integrations/webex/`) | Separate client for privileged operations (DECT, phone lookups, etc.) that require different scopes. |
|
||
| `src/integrations/xai/client.js` | All Grok/xAI calls. Contains both the short initial-description summarizer and the more detailed ticket summarizer. |
|
||
| `src/db/mappings.js` + `path.js` | Database access for work order ↔ room mappings. Path resolution is centralized here for safety. |
|
||
| `src/server/app.js` | Express routes (`/health`, `/webhook`, `/cleanup-test`, etc.). |
|
||
| `src/commands/*.js` | Individual slash command implementations. |
|
||
|
||
### Data & Action Flow Summary
|
||
|
||
- **Incoming webhook** → `server` → `webhookProcessor` → (DB + `webexService` + `xai`) → Webex room
|
||
- **User command** → Webex Framework (`bot/`) → specific command handler → (often external CollabSupport API) → response back to Webex
|
||
- **Space cleanup** → `cleanup-test` or scheduled job → `spaceCleanupService` → reads DB + ServiceChannel status → acts via `botClient`
|
||
|
||
All persistent state for work order rooms lives in the SQLite database (see "Database Constraint" below).
|
||
|
||
### What Happens When a Work Order Is Created (End-to-End Walkthrough)
|
||
|
||
Here is a concrete trace of what occurs when ServiceChannel sends a `WorkOrderCreated` webhook:
|
||
|
||
1. **Webhook arrives**
|
||
ServiceChannel calls `POST /webhook` with a payload containing `EventType: "WorkOrderCreated"` and the full work order object.
|
||
|
||
2. **Route hands off immediately**
|
||
`src/server/app.js` receives the request, logs it, returns `200 OK` quickly, and calls:
|
||
```js
|
||
webhookProcessor.processWebhook(payload)
|
||
```
|
||
|
||
3. **Concurrency protection**
|
||
`src/services/webhookProcessor.js` looks up (or creates) a Mutex for this specific `workOrderId`. It also maintains a small queue so multiple rapid events for the same work order are processed in order.
|
||
|
||
4. **Room lookup**
|
||
It queries the database (`src/db/mappings.js`) to see if a Webex room already exists for this work order.
|
||
|
||
5. **Room creation (first time only)**
|
||
If no room exists:
|
||
- Calls `webexService.createWorkOrderRoom(workOrder, teamId)`
|
||
- `webexService` calls `botClient.createRoom(...)` with the standard title format:
|
||
`ServChan WO-123456 | Store 2477 | Store Name`
|
||
- Adds the default members (the list currently hardcoded in `index.js`)
|
||
- Saves the mapping (`workOrderId → roomId`) to the database
|
||
|
||
6. **Description summarization**
|
||
Because this is a `WorkOrderCreated` event, it calls:
|
||
```js
|
||
summarizeTicketDescription(description, xaiToken)
|
||
```
|
||
from `src/integrations/xai/client.js`. This sends a focused prompt to Grok to turn the often messy ServiceChannel description into a clean 2–6 sentence summary.
|
||
|
||
7. **Message construction**
|
||
The processor builds a rich Markdown message containing:
|
||
- Link to the work order in ServiceChannel
|
||
- Store and trade information
|
||
- Priority, category, problem code
|
||
- Status
|
||
- The AI-generated description summary
|
||
|
||
8. **Message posted**
|
||
Calls `webexService.sendMarkdown(roomId, text)`, which ultimately calls `botClient.sendMarkdown(...)`.
|
||
|
||
9. **Cleanup**
|
||
The mutex is released. If the queue for this work order is now empty, the mutex and queue objects are removed from memory.
|
||
|
||
The result is a new, populated Webex space ready for the team, usually within a few seconds of the work order being created in ServiceChannel.
|
||
|
||
## Directory Structure
|
||
|
||
```
|
||
ServChan/
|
||
├── index.js # Thin bootstrap / orchestrator
|
||
├── src/
|
||
│ ├── bot/ # Webex Framework + command routing
|
||
│ ├── server/ # Express app (routes + handlers)
|
||
│ ├── services/ # Core business logic (webhookProcessor, webexService, etc.)
|
||
│ ├── integrations/ # External system clients (webex, serviceChannel, xai)
|
||
│ ├── db/ # Database access + safe path resolution
|
||
│ ├── commands/ # Individual slash command implementations
|
||
│ ├── legacy/ # Old code kept during transition (avoid editing)
|
||
│ ├── experimental/ # Superseded device code (mostly unused now)
|
||
│ └── utils/
|
||
├── tools/ # Standalone operational scripts
|
||
├── config/ # Non-secret config (secrets still here too)
|
||
├── data/ # SQLite database (handle with care)
|
||
├── Dockerfile
|
||
├── docker-compose.yml
|
||
├── REFACTOR-LOG.md # History of major changes (reference only)
|
||
└── README.md
|
||
```
|
||
|
||
## Operational Considerations
|
||
|
||
### The Database Constraint (Very Important)
|
||
|
||
The SQLite database file used by the live production bot must **never** be moved, renamed, or have a second copy accidentally created.
|
||
|
||
- All code resolves the DB path through `src/db/path.js`, which respects `DB_PATH` from your environment.
|
||
- The main application creates **one** connection in `index.js` and passes that instance around.
|
||
- When debugging, always verify which database file is actually being used (check logs at startup — it prints the path).
|
||
|
||
If you ever see two different `.db` files being touched, stop and investigate immediately.
|
||
|
||
### Remote Service Dependency
|
||
|
||
Many user-facing commands (`/avStatus`, `/woSummary`, `/woAttachments`, `/woHistory`) do **not** do the heavy lifting locally. They call an external service via the `CS_API_BASE` environment variable (currently pointing at a CollabSupport instance).
|
||
|
||
If those commands start failing but the webhook automation still works, the problem is usually with the remote service or network connectivity to it.
|
||
|
||
### Webex Token Separation
|
||
|
||
- Normal bot actions (creating rooms for work orders, posting messages, adding members) use the bot token via `botClient`.
|
||
- Advanced lookups (DECT networks, phones assigned to stores, etc.) require a different privileged token and go through `adminClient`.
|
||
|
||
Keep these two concerns separate. Do not mix the tokens.
|
||
|
||
### Logging & Observability
|
||
|
||
- Application logs are written daily to `./logs/YYYYMMDD.log`.
|
||
- The good structured logger lives in `src/utils/logger.js`.
|
||
- Some older legacy paths still use simpler console + file logging.
|
||
- The `/health` endpoint is your best friend for runtime state (DB connectivity, memory, uptime).
|
||
|
||
### Space Cleanup
|
||
|
||
Automated (or manual via `/cleanup-test`) removal of old completed work order spaces is handled by `spaceCleanupService.js`.
|
||
|
||
It reads work order status from ServiceChannel and then acts on the Webex rooms via the bot client. It is intentionally conservative.
|
||
|
||
## Running the Application
|
||
|
||
### Local
|
||
|
||
```bash
|
||
npm run dev
|
||
```
|
||
|
||
### Docker (recommended for consistency)
|
||
|
||
**Development** (live reload, source mounted, node --watch):
|
||
|
||
```bash
|
||
docker compose up --build
|
||
```
|
||
|
||
**Production** (optimized image, no source mount, persistent logs volume):
|
||
|
||
```bash
|
||
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d --build
|
||
```
|
||
|
||
After the first build you can usually omit `--build`:
|
||
|
||
```bash
|
||
docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d
|
||
```
|
||
|
||
See [docker-compose.prod.yml](docker-compose.prod.yml) for the production-specific settings.
|
||
|
||
**Important for production images**:
|
||
- All secrets must come from `.env` (via `env_file`). The application now enforces this via `src/config/secrets.js`.
|
||
- `config/config.json` must **not** contain real credentials (it can still hold non-secret values like server name/port or legacy non-token webex fields for compatibility).
|
||
- The production stage runs as the non-root `node` user (uid 1000). On Linux hosts ensure your `./data` and `./logs` host directories are writable by uid 1000 if you see permission errors.
|
||
|
||
### Production Considerations
|
||
|
||
- Use the production compose files for any long-lived or customer-facing deployment.
|
||
- Logs and the SQLite DB are the only things that need to persist across container restarts/recreates. Everything else is in the image.
|
||
- The `/health` and `/healthz` endpoints are suitable for container orchestrators (Kubernetes liveness/readiness, etc.).
|
||
- Consider adding resource limits, secrets management (Docker secrets / Vault), and log rotation outside the container for full prod ops.
|
||
- Always run `docker compose ... config` to validate before deploying.
|
||
|
||
### Health Checks
|
||
|
||
- `GET /healthz` → Simple liveness (used by Docker healthcheck)
|
||
- `GET /health` → Detailed JSON report including database connectivity and memory usage
|
||
|
||
### Common Debugging Commands
|
||
|
||
- Check which DB file is in use → Look at the very first logs after startup.
|
||
- Test space cleanup logic → `GET /cleanup-test?dryRun=true`
|
||
- Force a fresh xAI summary on a new work order → Create a test work order in ServiceChannel.
|
||
|
||
## Bot Commands
|
||
|
||
| Command | Context | Notes |
|
||
|------------------|----------------------------------|-------|
|
||
| `/help` | Anywhere | Context-aware |
|
||
| `/avStatus` | In WO space or with store number | Often calls remote service |
|
||
| `/woSummary` | In WO space or with WO number | Often calls remote service |
|
||
| `/woAttachments` | In WO space or with WO number | Often calls remote service |
|
||
| `/woHistory` | With store number | Often calls remote service |
|
||
| `/woApprove` | In WO space (or with WO#) | Manually posts proposal approval Adaptive Card (when status is IN PROGRESS \| WAITING FOR APPROVAL). Auto-triggered on relevant webhooks too. |
|
||
|
||
**Approval Cards (new)**: When a webhook arrives with `IN PROGRESS | WAITING FOR APPROVAL` (typically with a "Proposal created" note), ServChan automatically posts an Adaptive Card v1.3 in the room. The card shows proposal details/costs, prepopulates the suggested new NTE (current + proposal sums), and allows submit to PATCH the NTE in ServiceChannel + record approver attribution from the Webex user. Use `/woApprove` to re-trigger.
|
||
|
||
## Development Guidelines
|
||
|
||
When making changes:
|
||
|
||
- Prefer dependency injection (especially passing the `db` instance) over creating new connections.
|
||
- If you need to talk to Webex for normal bot operations, go through `webexService` rather than calling `botClient` directly.
|
||
- All AI summarization should live in or be called from `src/integrations/xai/`.
|
||
- Respect the production database path resolution — never hardcode a `.db` path.
|
||
- If you're touching webhook behavior, be extremely careful with concurrency (the mutex/queue pattern exists for a reason).
|
||
|
||
## Environment Variables
|
||
|
||
| Variable | Purpose | Notes |
|
||
|-------------------------|----------------------------------------------|-------|
|
||
| `DB_PATH` | SQLite database location | Critical — production uses this |
|
||
| `WEBEX_BOT_TOKEN` | Token for normal bot operations | Used by botClient |
|
||
| `CS_API_BASE` | Base URL for remote CollabSupport service | Powers many commands |
|
||
| `WEBEX_BOT_PERSON_ID` | Used to identify the bot during cleanup | Prevents removing itself |
|
||
| `WEBEX_ADMIN_TOKEN` (or similar) | Token for privileged Webex calls | Used by adminClient (if configured) |
|
||
|
||
Secrets must be provided exclusively via environment variables (loaded by `src/config/secrets.js` and required at startup for core functionality). `config/config.json` should contain **only non-secret values**. The code no longer depends on secrets living in the JSON file.
|
||
|
||
---
|
||
|
||
This document is meant to be living. Update it when you add significant new behavior or change how major flows work. |