> For the complete documentation index, see [llms.txt](https://docs.sec1.io/user-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.sec1.io/user-docs/9-setup-instructions/jira.md). # QC Jira + Zephyr Essential Collector — Runbook This guide shows how to **deploy the QC collector, configure it against a customer's Jira Server / Data Center instance (with Zephyr Essential), and trigger every collection step manually via REST endpoints**. Use it for onboarding a new customer, debugging a sync, or validating data before enabling scheduled collection. {% hint style="info" %} **Architecture in one line:** one container that reads bugs, tests, and Zephyr test executions from a single Jira DC instance, normalizes them, and writes to MongoDB. The dashboard / API service reads from Mongo — the collector never talks to Jira on the API path. {% endhint %} *** ## 1. What the collector does | Step | Job name | What it fetches | Mongo collections it writes | | --------------------------------------- | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ | | **Project discovery** | `qc_project_discovery` | Every Jira project visible to the PAT | `qc_projects`, `qc_collector_state` | | **Field discovery** | `qc_field_discovery` | Customfield IDs (Sprint, Story Points, Zephyr Teststep) | `qc_field_map`, `qc_collector_state` | | **Issue sync** | `qc_issues_sync` | Bugs + Test issues with `expand=changelog,names,schema`. Stores raw payload, derives normalized view, status history, and link graph. | `qc_issues_raw`, `qc_issues`, `qc_issue_history`, `qc_issue_links`, `qc_project_state`, `qc_collector_state` | | **Zephyr sync** (runs after issue sync) | `qc_zephyr_sync` | Test cycles + executions from `/rest/zapi/latest/*`. Enriches `qc_issues.test.*` on linked Test issues. | `qc_test_cycles`, `qc_test_executions`, `qc_issues.test.*` (in place) | All 11 `qc_*` collections live in the shared `dev-sec-ops-db` MongoDB. {% hint style="warning" %} **The collector is read-only against Jira.** It does not write labels, comments, or escalation tickets back. If a customer asks for writeback, push back to the design discussion — it's intentionally out of scope. {% endhint %} *** ## 2. Prerequisites Before you start: 1. **Customer-provided:** * Jira Server or Data Center base URL (e.g. `https://stg-jira.nomura.com`) * A service-account **Personal Access Token (PAT)** with **Browse Projects** permission on every project to be ingested * Confirmation that Zephyr Essential is installed and licensed (test artifacts won't surface otherwise) * Network access — typically VPN or IP-allowlist to reach the customer's Jira 2. **On the SEC1 side:** * MongoDB reachable at `host.docker.internal:27018` (SSH tunnel to the shared dev/test Mongo) **OR** a production Mongo URI * Docker / Docker Compose (Compose v2) * This repo cloned and built (`mvn -DskipTests package`) — or the prebuilt image pulled *** ## 3. Environment variables — manual-only setup This config disables crons so you can drive the pipeline by hand the first time. Once you've verified the data, flip `QC_SCHEDULER_ENABLED=true` to let the schedulers take over. ### Required ```env SPRING_PROFILES_ACTIVE=prod SERVER_PORT=8080 # ── Mongo (live shared db) ─────────────────────────────────────── # $ in the password MUST be URL-encoded as %24 (Docker Compose interpolation eats $). MONGODB_URI=mongodb://admin:@host.docker.internal:27018/dev-sec-ops-db?authSource=admin MONGODB_DATABASE=dev-sec-ops-db # ── Jira connection ────────────────────────────────────────────── JIRA_BASE_URL=https://stg-jira..com JIRA_USERNAME= JIRA_API_TOKEN= JIRA_AUTH_TYPE=pat # Server/DC = "pat" (Bearer). Cloud would be "basic" (email+token), not currently supported in prod. JIRA_HEALTH_CHECK_ENABLED=true JIRA_FAIL_ON_HEALTH_CHECK_ERROR=true # ── QC pipeline toggle ─────────────────────────────────────────── QC_ENABLED=true QC_SCHEDULER_ENABLED=false # start with crons OFF; flip to true after verification QC_ZEPHYR_ESSENTIAL_ENABLED=true # turn off only if customer hasn't licensed Zephyr # ── Disable health checks for unrelated collectors so this container starts cleanly ── SONARQUBE_HEALTH_CHECK_ENABLED=false NEXUSIQ_HEALTH_CHECK_ENABLED=false SERVICENOW_HEALTH_CHECK_ENABLED=false GITLAB_HEALTH_CHECK_ENABLED=false JENKINS_HEALTH_CHECK_ENABLED=false ``` ### Optional — tuning knobs (defaults are sensible) ```env QC_HISTORY_RETENTION_DAYS=30 # TTL on qc_issue_history.createdAt QC_FULL_SYNC_INTERVAL_HOURS=6 # how often to drop the "updated >=" delta filter QC_DELTA_OVERLAP_MINUTES=5 # safety overlap on watermark QC_FIRST_SYNC_CLOSED_WINDOW_DAYS=180 # cold-start closed-issue lookback QC_PROJECT_CHUNK_SIZE=200 # max project keys per JQL chunk QC_RATE_LIMIT_RPM=200 # token-bucket cap; tune per customer's Jira admin limits # Cron expressions (only matter when QC_SCHEDULER_ENABLED=true) QC_PROJECT_DISCOVERY_CRON=0 0 */6 * * * QC_FIELD_DISCOVERY_CRON=0 0 3 * * * QC_ISSUE_SYNC_CRON=0 */10 * * * * # Project scoping — leave empty for "all visible projects" QC_PROJECT_ALLOWLIST= # e.g. PAY,AUTH,INFRA QC_PROJECT_DENYLIST= # e.g. SANDBOX,PLAYGROUND # Logging LOG_LEVEL=INFO APP_LOG_LEVEL=INFO MONGO_LOG_LEVEL=INFO REST_LOG_LEVEL=INFO ``` {% hint style="warning" %} **Docker Compose $-escaping:** if your Mongo password (or `JIRA_API_TOKEN`) contains a literal `$`, double it in `.env` (`$$`) **and** URL-encode it as `%24` in `MONGODB_URI`. The two values diverge intentionally — one is read raw by the JVM, the other is parsed as a URL. {% endhint %} *** ## 4. Deployment ### Option A — Production: prebuilt image via docker-compose The `qc-jira-collector` service is already defined in [docker-compose.yml](https://github.com/sec0ne/user-docs/blob/main/docs/docker-compose.yml). Drop a tailored env file at `.env.qc-jira-collector` and bring it up: ```bash docker compose --env-file .env.qc-jira-collector up -d qc-jira-collector # Watch boot docker logs -f devsecops-qc-jira-collector ``` You should see, within \~30 seconds: ``` ... io.secone.config.JiraHealthCheck - INFO - Connected to Jira as user: () ... io.secone.config.JiraHealthCheck - INFO - Connected to Jira version: 10.x (Server) ... io.secone.config.JiraHealthCheck - INFO - QC: PAT can see N project(s) via /rest/api/2/project ... io.secone.config.JiraHealthCheck - INFO - QC: M issue types total, K Zephyr-flavoured ... io.secone.config.JiraHealthCheck - INFO - QC: detected J Zephyr custom field(s) ... io.secone.config.QcTtlReconciler - INFO - QC TTL on qc_issue_history.createdAt already set to 2592000s (30 days) ``` If health check fails (`Cannot connect to Jira`, `Jira authentication failed`, etc.), the container will exit unless `JIRA_FAIL_ON_HEALTH_CHECK_ERROR=false`. Fix credentials / network and restart. ### Option B — Local development with a Dockerised Jira test stack For local iteration without touching a customer's instance, use [docker-compose.testing.yml](https://github.com/sec0ne/user-docs/blob/main/docs/docker-compose.testing.yml) which spins up a Jira DC 10.3 container, Postgres, and the collector. See [TESTING.md](https://github.com/sec0ne/user-docs/blob/main/docs/TESTING.md) for the wizard / Zephyr-plugin install steps. *** ## 5. Manual collection — REST endpoints Two ways to drive collection: * **The simple "do everything" trigger** (most common — same naming as other collectors) * **Per-step admin endpoints** under `/api/jira/qc/` (for debugging or partial runs) Port `8086` in both production and local testing. ### 5.0 The single "kick off everything" trigger **This is the endpoint most people want.** Mirrors the convention used by every other collector in this repo (`/api/v1/collector//collect`). ```bash curl -X POST http://localhost:8086/api/v1/collector/jira/collect ``` Returns immediately (async — the work continues in the background): ```json { "success": true, "message": "Jira QC collection started (projects + fields + issues + Zephyr)", "timestamp": "2026-05-26T12:43:21.123" } ``` Under the hood it runs, in order: **project discovery → field discovery → issue sync → Zephyr sync.** Monitor progress via: ```bash curl -s http://localhost:8086/api/jira/qc/status | jq ``` You'll see each job's `status` advance through `running` → `idle`, with `lastSuccessAt` timestamps updating. Total time depends on issue count — for Nomura-scale (\~100K issues) expect 10–20 minutes on a cold first run; subsequent calls take seconds because of the hash-skip delta. There's also a project-only variant if you only want to refresh the project list: ```bash curl -X POST http://localhost:8086/api/v1/collector/jira/collect-projects ``` {% hint style="info" %} **Both `/api/v1/collector/jira/collect*` endpoints are async** — they return 200 immediately and the work continues. The `/api/jira/qc/sync/*` endpoints in §5.2–5.5 are synchronous — they block until the cycle finishes. Use the async one in scripts and CI; use the sync ones when you're debugging and want a single-step return. {% endhint %} ### 5.1 Health & status ```bash # General Spring Actuator health (Mongo + Jira reachability) curl -s http://localhost:8086/actuator/health | jq # QC pipeline status — projects, field map size, last-run per job curl -s http://localhost:8086/api/jira/qc/status | jq ``` Expected on a fresh deploy: `qcEnabled: true`, `activeProjects: 0`, `fieldMapEntries: 0`, all jobs with `lastSuccessAt: null`. ### 5.2 Project discovery — populate `qc_projects` ```bash curl -X POST http://localhost:8086/api/jira/qc/discover/projects | jq ``` Returns: ```json { "seen": 287, "created": 287, "updated": 0, "archived": 0, "durationMs": 4231 } ``` Inspect what was discovered: ```bash curl -s http://localhost:8086/api/jira/qc/projects | jq '.[] | {projectKey, projectName, archived, enabled}' ``` If you want to **scope** which projects to ingest, either: * Set `QC_PROJECT_ALLOWLIST=PAY,AUTH` in env and restart, OR * Toggle individual projects via: ```bash curl -X POST 'http://localhost:8086/api/jira/qc/projects/SANDBOX/enabled?value=false' ``` ### 5.3 Field discovery — populate `qc_field_map` ```bash curl -X POST http://localhost:8086/api/jira/qc/discover/fields | jq ``` Returns the count of logical names mapped to customfield IDs (expected 2–8 depending on what's installed): ```json { "resolved": 3 } ``` View the resolved map: ```bash curl -s http://localhost:8086/api/jira/qc/field-map | jq '.active' # { # "agile.sprint": "customfield_10020", # "agile.storyPoints": "customfield_10028", # "test.steps": "customfield_12031" # } ``` If a Zephyr field you expect isn't resolved, see [§7 Troubleshooting](#7-troubleshooting). ### 5.4 Issue sync — populate `qc_issues_raw`, `qc_issues`, `qc_issue_history`, `qc_issue_links` This is the main workhorse. On a cold start it does a full pull; on subsequent calls it's delta-driven. ```bash curl -X POST http://localhost:8086/api/jira/qc/sync/issues | jq ``` Returns: ```json { "projects": 287, "issuesUpserted": 14821, "historyInserted": 38104, "linksUpserted": 1207, "errors": 0, "durationMs": 642100, "fullSync": true } ``` Time scales roughly linearly with issue count + JQL chunk size. For Nomura-scale (\~100K issues) expect the first cycle to run 10–20 minutes; subsequent delta cycles drop to seconds because the hash-skip path identifies unchanged payloads and skips derivation. {% hint style="info" %} **Zephyr sync runs automatically after issue sync** when the scheduler is on. To trigger it manually for verification, see 5.5. {% endhint %} ### 5.5 Zephyr sync — populate `qc_test_cycles`, `qc_test_executions`, enrich `qc_issues.test.*` ```bash curl -X POST http://localhost:8086/api/jira/qc/sync/zephyr | jq ``` Returns: ```json { "totalProjects": 287, "projectsProcessed": 87, "projectsLicenseSkipped": 0, "cyclesUpserted": 412, "executionsUpserted": 9241, "testIssuesEnriched": 6088, "errors": 0, "durationMs": 87420, "licenseDisabled": false } ``` If `licenseDisabled: true`, Zephyr Essential's REST API rejected with `errorId: 12` ("Please enter valid license"). The Jira-issue side still works (Test issues are ingested by the issue sync), but execution data won't be collected. Once the customer applies a Zephyr license: ```bash # Clear the cached "disabled" sentinel curl -X POST http://localhost:8086/api/jira/qc/sync/zephyr/reset-license # Then re-run curl -X POST http://localhost:8086/api/jira/qc/sync/zephyr | jq ``` ### 5.6 Health-check the collected data — `/stats` Use this anytime you want a one-shot answer to "did the data collection actually work?". No Jira call; pure Mongo aggregation. ```bash curl -s http://localhost:8086/api/jira/qc/stats | jq ``` Returns a complete snapshot: ```jsonc { "counts": { "qc_projects": 287, "qc_issues_raw": 14821, "qc_issues": 14821, "qc_issue_history": 38104, "qc_issue_links": 1207, "qc_field_map": 3, "qc_test_cycles": 412, "qc_test_executions": 9241, "qc_project_state": 287, "qc_collector_state": 4, "qc_gate_evaluations": 0 // API service writes here; 0 is fine on a fresh deploy }, "freshness": { "qc_project_discovery": { "lastSuccessAt": "...", "ageSeconds": 1834, "lastError": null, "lastRunDurationMs": 4231 }, "qc_field_discovery": { "lastSuccessAt": "...", "ageSeconds": 8421, "lastError": null, ... }, "qc_issues_sync": { "lastSuccessAt": "...", "ageSeconds": 142, "lastError": null, ... }, "qc_zephyr_sync": { "lastSuccessAt": "...", "ageSeconds": 142, "lastError": null, ... } }, "issueType": { "STORY": 6890, "BUG": 4102, "TASK": 2900, "TEST": 612, "SUB_TASK": 317, "OTHER": 0 }, "severity": { "MEDIUM": 9100, "HIGH": 3201, "LOW": 1820, "CRITICAL": 700 }, "status": { "CLOSED": 9412, "OPEN": 3008, "IN_PROGRESS": 2401 }, "testStatus": { "PASS": 4012, "FAIL": 822, "BLOCKED": 94, "UNEXECUTED": 313 }, "linkType": { "Relates": 982, "Tests": 401, "Blocks": 124 }, "sanity": { "rawMatchesDerived": true, // qc_issues_raw count == qc_issues count "hasHistory": true, // changelog parsing produced transitions "hasLinks": true, // link reconciliation found edges "hasTests": true, // Zephyr Test issues ingested "hasZephyrExecutions": true, // Zephyr API sync produced executions "allIssuesUnmapped": false, // RED FLAG if true — issuetype normalizer broke "allSeveritiesMedium": false // RED FLAG if true — priority normalizer broke } } ``` {% hint style="info" %} **Use the `sanity` block as your acceptance test.** All seven flags green = collection worked. Any `true` on `allIssuesUnmapped` / `allSeveritiesMedium`, or any `false` on the `has*` flags after a full run, means something needs investigation — see [§7 Troubleshooting](#7-troubleshooting). {% endhint %} The `issueType` / `severity` / `status` / `testStatus` / `linkType` distributions are useful for spot-checks too — if a customer expects "lots of bugs" but you're seeing 90% STORY, the `issuetype-map` config may need an override. ### 5.7 Re-derive without re-collecting — `/reprocess` If you change the normalizer (e.g. add a new severity bucket, fix a Zephyr field mapping), bump the `DERIVER_VERSION` constant in `JiraQcIssuesSyncService` and call: ```bash # Re-derive ONLY docs whose derivedVersion is stale (fast) curl -X POST 'http://localhost:8086/api/jira/qc/reprocess?onlyStale=true' | jq # Re-derive everything (slower; use after a deeper change) curl -X POST 'http://localhost:8086/api/jira/qc/reprocess?onlyStale=false' | jq ``` Returns: ```json { "totalRaw": 14821, "processed": 14821, "errors": 0, "durationMs": 38727, "deriverVersion": "v2" } ``` **This makes zero Jira API calls.** The customer's instance is untouched. Derivation runs against the locally-stored verbatim payloads in `qc_issues_raw`. ### 5.8 Wipe all collected data — `/wipe` {% hint style="danger" %} **DESTRUCTIVE.** This drops every `qc_*` collection in Mongo. Use only when: * Starting fresh against a new customer * Recovering from a bad ingestion run * Tearing down a test environment The endpoint refuses to run without an explicit `confirm=DELETE` query parameter — a hand-typed token automation won't accidentally produce. {% endhint %} ```bash # Refused without confirmation curl -X POST http://localhost:8086/api/jira/qc/wipe | jq # { # "error": "missing confirmation", # "hint": "Append ?confirm=DELETE to confirm. This drops ALL qc_* collections." # } # Confirmed — actually drops every qc_* collection curl -X POST 'http://localhost:8086/api/jira/qc/wipe?confirm=DELETE' | jq ``` Returns: ```json { "totalDocsDeleted": 64072, "collectionsDropped": 11, "perCollection": { "qc_projects": { "dropped": true, "docsBefore": 287 }, "qc_issues_raw": { "dropped": true, "docsBefore": 14821 }, "qc_issues": { "dropped": true, "docsBefore": 14821 }, "qc_issue_history": { "dropped": true, "docsBefore": 38104 }, "qc_issue_links": { "dropped": true, "docsBefore": 1207 }, "qc_field_map": { "dropped": true, "docsBefore": 3 }, "qc_test_cycles": { "dropped": true, "docsBefore": 412 }, "qc_test_executions": { "dropped": true, "docsBefore": 9241 }, "qc_project_state": { "dropped": true, "docsBefore": 287 }, "qc_collector_state": { "dropped": true, "docsBefore": 4 }, "qc_gate_evaluations": { "dropped": true, "docsBefore": 0 } }, "note": "Indexes will be recreated on next collector startup. To repopulate now, call POST /api/v1/collector/jira/collect" } ``` **Zero Jira calls** — only Mongo collections are dropped. The customer's Jira instance is untouched. Indexes are recreated automatically on the next collector startup (via `QcTtlReconciler`). To rebuild the data right away: ```bash curl -X POST http://localhost:8086/api/v1/collector/jira/collect ``` *** ## 6. Going from manual to scheduled Once the data looks right in Mongo (or in the dashboard): 1. Set `QC_SCHEDULER_ENABLED=true` 2. Restart the container 3. Verify in `/api/jira/qc/status` that `lastRunAt` advances on the configured cron cadence The default crons are conservative and won't overwhelm Jira: * Project discovery: every 6 hours * Field discovery: daily at 03:00 * Issue sync (with Zephyr appended): every 10 minutes — delta-driven, so steady-state cost is small *** ## 7. Troubleshooting | Symptom | Likely cause | Fix | | ----------------------------------------------------------------------- | --------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | | Container exits at startup with `Cannot connect to Jira at https://...` | Network / VPN / token | Verify the URL is reachable from inside the container: `docker exec qc-test-jira-collector curl -fsS https://stg-jira..com/rest/api/2/serverInfo` | | Health check passes but `QC: PAT can see 0 project(s)` | PAT doesn't have Browse Projects on any project | Have the customer scope the service account broader, or set `QC_PROJECT_ALLOWLIST` to a known-good project | | `Connection refused` on Mongo | SSH tunnel down OR password not URL-encoded | Check `nc -zv localhost 27018`; ensure `$` in password is `%24` in `MONGODB_URI` | | `licenseDisabled: true` from `/sync/zephyr` | Customer hasn't applied a Zephyr Essential license | Customer applies a license; call `/sync/zephyr/reset-license`, retry | | Issue sync returns `error: ... '%' is a reserved JQL character` | Re-encoding bug (resolved in current build) | Pull latest; this was a known double-encode that's now fixed | | `qc_issue_history` rows duplicating | Unique compound index missing | Check `db.qc_issue_history.getIndexes()` — should include `unique_transition`. If missing, restart the container; `QcTtlReconciler` will create it. | | Sync cycle takes >30 minutes | First cold start at Nomura-scale, or a slow Jira instance | Inspect `qc_collector_state.lastRunDurationMs`; lower `QC_PROJECT_CHUNK_SIZE` if individual JQL calls are timing out | | A Zephyr field we expect (e.g. test status) isn't resolved | Customer named it differently from our heuristic | Add a `QC_CUSTOM_FIELD_OVERRIDES` config entry, or extend the heuristic in `JiraFieldDiscoveryService` | | Spring binding error like `No setter found for property: qc-enabled` | Old env var name | Use `QC_*` (top-level) not `JIRA_QC_*` — that prefix was renamed | ### Useful one-liners ```bash # Last log lines docker logs --tail 100 devsecops-qc-jira-collector # Per-project sync state — useful when a single project is stuck curl -s http://localhost:8086/api/jira/qc/project-state | jq # Reset a project's cursor (forces re-pull on next cycle) curl -X POST http://localhost:8086/api/jira/qc/project-state/PAY/reset # Mongo inspect (data freshness) mongosh "$MONGODB_URI" --eval 'db.qc_collector_state.find({}, {jobName:1, lastSuccessAt:1, lastRunDurationMs:1, _id:0}).forEach(printjson)' # Mongo inspect (a sample issue) mongosh "$MONGODB_URI" --eval 'printjson(db.qc_issues.findOne({issueType: "BUG"}))' ``` *** ## 8. Endpoint reference ### "Kick off everything" — aggregate triggers (async) | Method | Path | Purpose | | ------ | ----------------------------------------- | -------------------------------------------------------------------------------------------------------------- | | `POST` | `/api/v1/collector/jira/collect` | **The main entry point.** Runs project + field discovery, issue sync, Zephyr sync — all in one background job. | | `POST` | `/api/v1/collector/jira/collect-projects` | Just refresh the project list (background) | ### Per-step admin endpoints (synchronous) All under `/api/jira/qc/`. | Method | Path | Purpose | | ------ | --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | | `GET` | `/status` | Overall health + per-job last-run stats | | `GET` | `/projects` | Discovered project list (filter `?archived=true` for archived) | | `POST` | `/projects/{key}/enabled?value={true\|false}` | Toggle a project's `enabled` flag | | `GET` | `/field-map` | Active customfield map + raw entries | | `GET` | `/project-state` | Per-project sync watermarks | | `GET` | `/project-state/{key}` | Single project watermark | | `POST` | `/project-state/{key}/reset` | Reset a project's cursor (force re-pull) | | `POST` | `/discover/projects` | Run project discovery now | | `POST` | `/discover/fields` | Run customfield discovery now | | `POST` | `/sync/issues` | Run an issue sync cycle now | | `POST` | `/sync/zephyr` | Run Zephyr cycles + executions sync now | | `POST` | `/sync/zephyr/reset-license` | Clear cached "Zephyr disabled" sentinel after a license is applied | | `POST` | `/reprocess?onlyStale={true\|false}` | Re-derive normalized from raw with **zero Jira calls** | | `GET` | `/stats` | Sanity snapshot: per-collection counts, freshness, distributions, sanity flags. Use as acceptance test after a run. | | `POST` | `/wipe?confirm=DELETE` | **DESTRUCTIVE.** Drops every `qc_*` collection (refuses without the confirm token). Used for clean restarts. | *** ## 9. Companion docs * [TEST\_RESULTS.md](https://github.com/sec0ne/user-docs/blob/main/docs/TEST_RESULTS.md) — what was verified end-to-end against a local Jira + Zephyr Essential stack * [QC\_API\_HANDOFF.md](https://github.com/sec0ne/user-docs/blob/main/docs/QC_API_HANDOFF.md) — schema reference for the API service that reads the Mongo collections * [CLAUDE.md](https://github.com/sec0ne/user-docs/blob/main/docs/CLAUDE.md) — architectural decisions and conventions for this codebase * [TESTING.md](https://github.com/sec0ne/user-docs/blob/main/docs/TESTING.md) — how to bring up the local Jira test stack