Zoomed Image
USA
800 691 9120
UK
01225 704844

Docs day-long audit — summary

Docs day-long audit — summary

Docs day-long audit — summary

What was built (tooling, end-to-end pipeline)

The docs/screenshot pipeline is now driven from <!-- robot: ... --> comments embedded in each .md file. All artifacts are regenerated by the website on startup when _Layout.cshtml is touched (DEBUG mode), and by a single set of generators in the Website project.

Artifact Generator Purpose
help/feature-catalog.json FeatureCatalogGenerator Every menu × profile × category × resolved target. Source of truth for "what menu names exist today". Global Version 7 is merged into each user-facing profile so each profile reads as self-contained.
help/query-columns.json QueryColumnsGenerator Every query's on-screen columns in display order. Used to cross-check that a recipe lands on the screen the prose describes.
help/robot-manifest.json RobotManifestGenerator The Robot input. Generated from the MD comments — comments are now the source of truth, not the manifest.

Robot enhancements:

  • Failure logging — TestScript.cs now tracks // <id>: <desc> markers from ManifestScriptBuilder and accumulates per-screenshot failures. After a docs run, writes help/screenshot-failures.md listing each ID where a step threw, the line number, and the error.
  • Copy step — ManifestScriptBuilder.CopyCapturesToHelp(sinceUtc) runs after the docs script finishes and copies fresh PNGs (filtered by mtime > run-start) into help/images/[output]. Downstream: Website.DocumentationSync.Sync propagates to wwwroot/docs-data/, and WordToHelp md propagates to customerdata/x7/help/.

What was audited

112 pages processed by 4 parallel Sonnet agents (one per guide), then a focused page-action follow-up agent.

Guide Files Edits applied
itam-guide 53 (50 + 3 pilot) ~97
sam-guide 17 ~42
fam-guide 20 7
maintenance-guide 13 9
barcoding-guide 5 2
keywedge-guide 3 0
Page-action follow-up 22 (subset) 5
Total 112 pages ~183 edits, 213 flags

Per-page reports under help/docs-mismatch-reports/ (gitignored — regeneratable).

The page-action menu unlock

Most "manual" flags before today were because Robot's ClickMenu couldn't reach items inside toolbar dropdowns ("Software Licensing" button on Licensing Position, "Actions on Selected Records", "Edit", etc.). Inspecting the live UI revealed the page-action toolbar lives in #centremenu. Using ClickMenu's optional second arg as a scope unlocks the pattern:

ClickMenu "Licensing Position"
| ClickMenu "Software Licensing" "#centremenu"
| ClickMenu "Create a new Software Catalog Entry"
| PrintScreen "id"

Saved as reference_robot_page_action_menus.md in memory so future docs work doesn't relearn this.

The follow-up agent converted 5 manual flags to working page-action recipes; another ~10 confirmed manual flags (asset-editor states, transient dialogs) can never be reached this way.

Manifest health (before / after today)

Before (Mar 13 baseline) After today
Manifest entries 256 504
Automatable ~256 (but many failing silently) 188
Manual / not-automatable 0 explicit 316 (with reasons)
id-only-only recipes ~120 20
id-only-missing 42 3

The reduction in "automatable" reflects honesty, not regression — recipes that were silently failing (capturing the home page) are now correctly flagged manual with reasons. The 504 vs 256 increase is because many existing comments referenced screenshots that the old manifest didn't carry; agents brought them all into one source.

Top recurring patterns (from reports)

  • Bare-ID recipes — 82+ in itam-guide alone, all converted to form-2 or manual
  • Page-action menus — flagged in 32 reports
  • Transient/dialog states — flagged in 35 reports (genuinely not automatable)
  • Sidebar category headers — flagged in 13 reports (likely already work; needs spot-check)
  • "Software Product Catalog" → "Licensing Position" rename applied across multiple files
  • "Disposed Inventory" → "Disposed Assets" rename applied to fam-guide
  • Catalog typo "Disposal and Obsolescense" — recipes match the typo so ClickMenu finds the menu; prose stays correctly spelled

Most-flagged pages (need human review)

These pages had ≥4 items flagged — too many ambiguities for the agents to safely touch:

  • fam-guide/section-2-menus/classification-menu.md (6)
  • itam-guide/change-reporting/email-notifications.md (4)
  • itam-guide/contracts/overview.md (4)
  • sam-guide/software-catalog/defining-titles.md (4)
  • itam-guide/software-licencing/software-licences.md (4)
  • itam-guide/asset-management/disposal.md (4)

Full ranked list in help/docs-mismatch-master.md.

What's still open

  • ~316 manual recipes — a portion are genuinely manual (transient dialogs, asset-editor contexts, hardware shots) and won't ever be Robot-driven; the rest are candidates for a future pattern unlock similar to #centremenu.
  • Visual review of new captures — Robot's failure log catches ClickMenu-not-found errors but NOT recipes that ran cleanly but landed on the wrong screen. That class still needs eyes. Suggest: spot-check 10–20 of the most-changed pages against rendered docs.
  • Content rewrites flagged for human (e.g. "discovery handles this automatically — section is stale") — see the per-page reports' "Flagged for human review" sections.
  • Configuration-guide and procurement-guide were skipped today (low audit yield without a clear profile mapping).

How to keep it healthy

When a menu is renamed in the product:

  1. Touch _Layout.cshtml and restart the website in DEBUG → all three artifacts regenerate.
  2. The bare regeneration won't fix prose drift; spawn a Sonnet agent on affected pages with the standard audit prompt (template in software-products.md / disposal.md reports as worked examples).
  3. Run Robot Option 5 → captures propagate, failures logged.