{"id":141,"date":"2024-11-07T16:21:39","date_gmt":"2024-11-07T12:21:39","guid":{"rendered":"https:\/\/artenatech.com\/?page_id=141"},"modified":"2026-08-24T21:23:55","modified_gmt":"2026-08-24T17:23:55","slug":"skillforge","status":"publish","type":"page","link":"https:\/\/artenatech.com\/index.php\/skillforge\/","title":{"rendered":"SkillForge \u2014 Platform for Agent Skills, Workflows & Verification Hooks"},"content":{"rendered":"\n

SkillForge <\/strong>is a centralized platform for managing production-grade agent capabilities: composable skills, multi-step workflows, verification hooks, and autonomous routines. Unlike static prompt repositories, SkillForge is built on ablation-first principles \u2014 every instruction must justify itself through evals, every skill has a shelf life, and the system continuously pushes you to delete rather than add.<\/p>\n\n\n\n

\"\"<\/figure>\n\n\n\n

Inspired by how frontier agent harnesses are rebuilt every model generation, SkillForge treats agent capabilities as disposable building blocks. Your team gets a unified registry where skills are continuously tested, measured for product overhang, and retired when models outgrow them. The goal is not to accumulate instructions, but to unhobble the model and let it do what it already can.<\/p>\n\n\n\n

What SkillForge Is \u2014 and What It Is Not<\/strong><\/h3>\n\n\n\n

The previous generation of tooling stored prompts<\/strong>: free-text guesses about what a model might need. Prompts have no evidence, no ownership, no lifecycle. They accumulate, rot, and silently tax every run with tokens the model no longer reads. SkillForge stores skills<\/strong>: named, versioned, gate-proven procedures with triggers, verification hooks, eval results, and provenance. A skill is not a hope \u2014 it is a capability with a track record.<\/p>\n\n\n\n

<\/th>Prompt registry<\/th>SkillForge<\/strong><\/th><\/tr><\/thead>
Unit of storage<\/td>Free-text prompt<\/td>SKILL.md procedure with frontmatter contract<\/strong><\/td><\/tr>
Evidence<\/td>None<\/td>Gate-proven runs, eval pass rates, provenance stamps<\/strong><\/td><\/tr>
Lifecycle<\/td>Forever (until someone notices)<\/td>Born \u2192 Active \u2192 Aging \u2192 Zombie \u2192 Retired<\/strong><\/td><\/tr>
Quality signal<\/td>Human opinion<\/td>Ablation Score, pass rates, transfer efficiency<\/strong><\/td><\/tr>
Safety<\/td>Hope<\/td>Verification hooks: pre \/ mid \/ post execution<\/strong><\/td><\/tr>
Composition<\/td>Copy-paste<\/td>Workflows: sequential, parallel, fan-out\/fan-in<\/strong><\/td><\/tr>
Runtime<\/td>One tool, one format<\/td>Cross-agent: SKILL.md open standard, MCP, REST<\/strong><\/td><\/tr>
Default action<\/td>Add<\/td>Delete (ablation-first)<\/strong><\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
\n\n\n\n

The Philosophy: Ablation-First<\/strong><\/h3>\n\n\n\n

SkillForge operationalizes the discipline that frontier harness teams practice by hand:<\/p>\n\n\n\n

    \n
  • Delete first.<\/strong> Every model generation, large parts of the system prompt, the tool set, and the skill library stop earning their place. The correct default is deletion; additions must prove themselves.<\/li>\n\n\n\n
  • Skills expire.<\/strong> A skill written for yesterday’s model may hobble today’s model. Plan to retire, not just maintain.<\/li>\n\n\n\n
  • Evals are the source of truth \u2014 until they saturate.<\/strong> Prompts and skills are guesses; evals are measurements. But evals also outlive their usefulness: when a model saturates an eval, the eval is thrown away and a harder one is built.<\/li>\n\n\n\n
  • Unhobble, don’t constrain.<\/strong> Product overhang is real \u2014 models can do more than we think, and over-specified instructions get in the way. The registry measures the gap between what your agents can<\/em> do and what they’re allowed<\/em> to do.<\/li>\n<\/ul>\n\n\n\n

    The platform’s home screen asks one question: “What can you delete today?”<\/strong><\/p>\n\n\n\n

    \n\n\n\n

    How It Works \u2014 Core Capabilities<\/strong><\/h3>\n\n\n\n

    1. Ablation-First Registry<\/strong><\/h4>\n\n\n\n

    The registry is an instrument, not a folder. It continuously measures whether each skill still matters.<\/p>\n\n\n\n

      \n
    • Versioned skills tied to model generations<\/strong> \u2014 every skill records which model generation it was born on and which generations it has been validated against (“written for Opus 4.8, untested on Opus 5 \u2014 ablation recommended”)<\/li>\n\n\n\n
    • Automated ablation tests<\/strong> \u2014 the eval suite runs with and without the skill; the delta is the skill’s measured contribution<\/li>\n\n\n\n
    • Ablation Score<\/strong> \u2014 the percentage of your library that can be deleted without quality loss; the overengineering meter<\/li>\n\n\n\n
    • Aging alerts<\/strong> \u2014 skills not validated in the last 90 days are flagged for review<\/li>\n\n\n\n
    • New-model release triggers<\/strong> \u2014 when a new model generation lands, SkillForge queues an ablation sweep across the library and reports which skills became dead weight overnight<\/li>\n<\/ul>\n\n\n\n

      2. Skill Lifecycle Management<\/strong><\/h4>\n\n\n\n

      Every skill moves through an evidence-driven lifecycle:<\/p>\n\n\n\n

        \n
      • Born<\/strong> \u2014 created from a proven source: a gate-proven Reckon run, an AgentSpace demonstration, a Dream Team certification, or a human-authored procedure that passed its validation gate. No proof \u2192 no skill.<\/li>\n\n\n\n
      • Active<\/strong> \u2014 injected on trigger match; every injection tracks uses<\/code> and helped<\/code> (the run it was shown to then passed its gate). Proof-of-benefit outranks proof-of-selection everywhere.<\/li>\n\n\n\n
      • Aging<\/strong> \u2014 unused candidates are pruned; unused active skills are demoted back to candidate \u2014 never deleted<\/strong> if promoted. Using or helping a skill reinforces it and resets its clock.<\/li>\n\n\n\n
      • Zombie<\/strong> \u2014 the model outgrew it: ablation shows zero delta, yet the skill still loads and still costs tokens on every run. Flagged for retirement.<\/li>\n\n\n\n
      • Retired<\/strong> \u2014 archived with full audit trail and eval history; restorable if a future model generation regresses the capability.<\/li>\n<\/ul>\n\n\n\n

        Side-by-side comparison shows a skill’s effectiveness across model versions, and rollback is one click.<\/p>\n\n\n\n

        3. Verification Hooks (First-Class Citizens)<\/strong><\/h4>\n\n\n\n

        Hooks are the safety and verification layer of the stack \u2014 and they are treated as product, not config.<\/p>\n\n\n\n

          \n
        • Pre-execution hooks<\/strong> \u2014 input validation, context checks, safety gates. Modeled on the balanced-safety philosophy: default to ask<\/code>, not deny<\/code><\/strong> \u2014 agents trivially bypass deny<\/code> by rephrasing, so hooks interrupt only when an action is genuinely destructive (catastrophic paths: recursive deletes, force-pushes, infra mutations, DB clients, cloud control-plane changes), with safe-path carve-outs to keep false positives near zero.<\/li>\n\n\n\n
        • Mid-execution checkpoints<\/strong> \u2014 the agent self-verifies along the way: the single most important thing you can give an agent is a way to check its own work.<\/li>\n\n\n\n
        • Post-execution validation<\/strong> \u2014 outputs checked against eval suites and structural scanners; regressions raise alerts.<\/li>\n\n\n\n
        • Human-in-the-loop approval gates<\/strong> \u2014 sensitive operations pause for a named human with full context.<\/li>\n\n\n\n
        • Cross-agent hook protocol<\/strong> \u2014 the same hooks run in Claude Code, Codex CLI, OpenCode, and any runtime supporting the hook standard; compiled-binary hooks for the cases that can’t live in skill format.<\/li>\n<\/ul>\n\n\n\n

          4. Workflow<\/strong> Composer<\/strong><\/h4>\n\n\n\n

          A visual algebra for orchestrating agents at scale:<\/p>\n\n\n\n

            \n
          • Chain skills into multi<\/strong>-step workflows with conditional branching and error-recovery paths<\/li>\n\n\n\n
          • Visual workflow editor for non<\/strong>-technical stakeholders<\/li>\n\n\n\n
          • Debug mode<\/strong> with step-by-step execution traces<\/li>\n\n\n\n
          • Replay failures for root-cause analysis<\/strong> \u2014 any historical run can be re-executed step by step<\/li>\n<\/ul>\n\n\n\n

            5. Routines Engine<\/strong><\/h4>\n\n\n\n

            Cron-like autonomous tasks that keep systems healthy without shared context:<\/p>\n\n\n\n

              \n
            • “Abstraction police”<\/strong> \u2014 find near-duplicate abstractions across codebases and unify them<\/li>\n\n\n\n
            • Dead-code cleanup<\/strong>, stale-test removal, test-coverage automation<\/li>\n\n\n\n
            • Experiment shipping<\/strong> \u2014 promote fully-ramped experiments and delete their flags<\/li>\n\n\n\n
            • Runs without shared context, with persistent memory<\/strong>; each run is isolated and auditable<\/li>\n\n\n\n
            • Built-in monitoring, failure recovery, and budget caps<\/li>\n<\/ul>\n\n\n\n

              6. Product Overhang Detector<\/strong><\/h4>\n\n\n\n

              The unhobbling instrument:<\/p>\n\n\n\n

                \n
              • Compares agent behavior with and without<\/strong> each skill and instruction set<\/li>\n\n\n\n
              • Identifies capabilities blocked by over-specification (“your skill restricts the agent to snippets; the model can now write entire modules”)<\/li>\n\n\n\n
              • Quantifies the gap between what models can do and what they’re allowed to do<\/li>\n\n\n\n
              • Issues unhobbling recommendations<\/strong>: remove this skill, re-run the eval suite, keep the deletion if the delta is zero<\/li>\n<\/ul>\n\n\n\n

                7. Eval-Driven Quality Gates<\/strong><\/h4>\n\n\n\n

                Every skill is tied to evidence:<\/p>\n\n\n\n

                  \n
                • Automated eval suites<\/strong> per skill, run on real production-shaped tasks<\/li>\n\n\n\n
                • Pass rates<\/strong>, token efficiency<\/strong>, and cost-per-success<\/strong> tracked per skill per model generation<\/li>\n\n\n\n
                • Eval versioning<\/strong> \u2014 evals are first-class artifacts that outlive skills by 2\u20133 generations<\/li>\n\n\n\n
                • Saturation detection<\/strong> \u2014 when a model starts maxing an eval, the platform flags it: the eval has stopped measuring; build a harder one and retire the old<\/li>\n<\/ul>\n\n\n\n

                  8. Workflow<\/strong> Orchestration<\/strong><\/h4>\n\n\n\n

                  Skills compose into pipelines:<\/p>\n\n\n\n

                    \n
                  • Chain skills into multi-step workflows with conditional branching and error-recovery paths<\/li>\n\n\n\n
                  • Visual workflow editor for non-technical stakeholders<\/li>\n\n\n\n
                  • Debug mode with step-by-step execution traces<\/li>\n\n\n\n
                  • Replay failures for root-cause analysis \u2014 any historical run can be re-executed step by step<\/li>\n<\/ul>\n\n\n\n

                    9. Team Collaboration & Governance<\/strong><\/h4>\n\n\n\n
                      \n
                    • Share skills across teams with granular permissions<\/strong> (private \/ team \/ org \/ public registries)<\/li>\n\n\n\n
                    • Review workflows and approval gates<\/strong> for skill changes; nothing reaches Active without a named reviewer<\/li>\n\n\n\n
                    • Comment threads<\/strong> on skill effectiveness; ratings grounded in eval data, not vibes<\/li>\n\n\n\n
                    • Templates<\/strong> for common patterns: safety, verification, error handling, incident response<\/li>\n\n\n\n
                    • Full audit log<\/strong> of every modification, promotion, demotion, and retirement<\/li>\n\n\n\n
                    • SkillOpt-style optimization loop<\/strong> for improving skills safely: bounded edits, a held-out validation gate, a rejected-edit buffer, and epoch-wise slow\/meta updates \u2014 skills improve by measured increments, never by vibes<\/li>\n<\/ul>\n\n\n\n

                      10. Universal Agent Runtime Integration<\/strong><\/h4>\n\n\n\n

                      Skills follow the Agent Skills open standard<\/strong> (SKILL.md), so one registry serves the whole fleet:<\/p>\n\n\n\n

                        \n
                      • Native support for Claude Code, Cursor, Codex, Windsurf, Copilot, Gemini CLI, OpenCode, Antigravity<\/strong> \u2014 anywhere the standard reaches<\/li>\n\n\n\n
                      • MCP server<\/strong> for remote skill discovery and injection<\/li>\n\n\n\n
                      • REST API and WebSocket<\/strong> support for custom integrations and CI<\/li>\n\n\n\n
                      • Hot-swap<\/strong> skills without restarting agent sessions<\/li>\n\n\n\n
                      • One-command install and sync across runtimes (skills add<\/code> style), with per-runtime config differences handled automatically<\/li>\n<\/ul>\n\n\n\n
                        \n\n\n\n

                        Skill Anatomy<\/strong><\/h3>\n\n\n\n

                        A SkillForge skill is a contract, not a note:<\/p>\n\n\n\n

                          \n
                        • Frontmatter<\/strong> \u2014 name, version (semver), triggers, model compatibility, provenance stamp (source run \/ episode \/ demonstrator \/ author), linked evals, linked hooks, trust level<\/li>\n\n\n\n
                        • Procedure<\/strong> \u2014 the steps that worked, including the step most likely to be missed<\/li>\n\n\n\n
                        • References \/ scripts \/ assets<\/strong> \u2014 supporting material, kept out of the always-on context<\/li>\n\n\n\n
                        • Trigger model<\/strong> \u2014 library-relative IDF: a trigger recurring across the whole library is discounted, so injection fires on discriminating domain signal<\/strong>, not boilerplate; the \u22652-word bar means “two words’ worth of meaningful<\/em> signal.” Self-tunes per project, no domain nouns hard-coded<\/li>\n\n\n\n
                        • Evidence links<\/strong> \u2014 the run’s diff, its compaction segments, its eval results: any skill can be traced back to the verbatim ground truth that produced it<\/li>\n<\/ul>\n\n\n\n

                          Token-cheap by design: skill descriptions are kept tight so they never dominate context; heavy detail lives in references the agent pulls only when needed.<\/p>\n\n\n\n

                          \n\n\n\n

                          Where Skills Come From \u2014 the Provenance Pipeline<\/strong><\/h3>\n\n\n\n

                          SkillForge is the convergence point of the whole stack’s learning:<\/p>\n\n\n\n

                            \n
                          • From Reckon<\/strong> \u2014 every run that passes its objective gate may distil a candidate procedure (no proof \u2192 no skill). Candidates stay dormant until a human promotes them<\/strong>; only promoted skills are ever injected, and only when their trigger matches.<\/li>\n\n\n\n
                          • From AgentSpace<\/strong> \u2014 “Watch Me” demonstrations become replayable, named skills with decision points, not fragile macros.<\/li>\n\n\n\n
                          • From Dream Team<\/strong> \u2014 certified patterns and consilium-validated procedures graduate into the registry with confidence scores.<\/li>\n\n\n\n
                          • From humans<\/strong> \u2014 expert procedures enter through a validation gate: they must pass evals before they can steer agents.<\/li>\n<\/ul>\n\n\n\n

                            One rule governs all four sources: a skill without evidence is a candidate, and a candidate is inert.<\/strong><\/p>\n\n\n\n

                            \n\n\n\n

                            Metrics That Matter<\/strong><\/h3>\n\n\n\n
                            Metric<\/th>What it measures<\/th>Why it exists<\/th><\/tr><\/thead>
                            Ablation Score<\/strong><\/td>% of library deletable without quality loss<\/td>Overengineering meter<\/td><\/tr>
                            Skill Age<\/strong><\/td>Time since last validation<\/td>Rot control<\/td><\/tr>
                            Verification Pass Rate<\/strong><\/td>% of hook checks passed<\/td>Safety health<\/td><\/tr>
                            Transfer Rate<\/strong><\/td>Injections that led to a passing run (helped<\/code> \/ uses<\/code>)<\/td>Proof-of-benefit ranking<\/td><\/tr>
                            Product Overhang Index<\/strong><\/td>Gap between “can do” and “allowed to do”<\/td>Unhobbling backlog<\/td><\/tr>
                            Eval Saturation<\/strong><\/td>Models maxing an eval<\/td>Signal to rebuild evals<\/td><\/tr>
                            Token Efficiency<\/strong><\/td>Tokens per successful task, per skill<\/td>Cost of carrying a skill<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n
                            \n\n\n\n

                            Real-World Use Cases<\/strong><\/h3>\n\n\n\n

                            Case 1: The Post-Upgrade Purge.<\/strong> After a new model generation, an enterprise ran SkillForge’s ablation sweep: 61% of a 400-skill library showed zero delta. Retiring them cut prompt tokens by 44% per run and raised<\/em> pass rates \u2014 the deleted instructions had been hobbling the new model.<\/p>\n\n\n\n

                            Case 2: Safety Without Friction.<\/strong> A fleet running destructive-capable agents adopted balanced verification hooks: catastrophic paths ask, safe paths never interrupt. False-positive interruptions stayed under 2%, while three would-be catastrophic operations (force-push to main, recursive delete, prod infra mutation) were caught in one quarter.<\/p>\n\n\n\n

                            Case 3: Incident Response Workflow<\/strong>. A workflow chains detection \u2192 triage \u2192 remediation skills with a human approval gate before any rollback. Mean time to remediate fell 58%; every step is replayable for postmortems.<\/p>\n\n\n\n

                            Case 4: Routines as Maintenance Crew.<\/strong> Abstraction-police and dead-code routines run nightly across 14 repositories, opening 20\u201330 small, reviewable pull requests a day. Engineers review; agents maintain.<\/p>\n\n\n\n

                            \n\n\n\n

                            Integration with the Artena Stack<\/strong><\/h3>\n\n\n\n

                            Reckon \u2192<\/strong> gate-proven learned skills flow into SkillForge; promoted skills are injected back into Reckon runs on trigger match. The honest-gate rule applies to learning itself.<\/p>\n\n\n\n

                            AgentSpace \u2192<\/strong> demonstration learning publishes skills; fleet agents consume them on every task; usage and transfer metrics flow back for curation.<\/p>\n\n\n\n

                            Dream Team \u2192<\/strong> certification scenarios validate skills against new model generations; retired skills become training material on “what we used to need.”<\/p>\n\n\n\n

                            \n\n\n\n

                            Core Principles<\/strong><\/h3>\n\n\n\n
                              \n
                            • Delete more, build less<\/strong>. Every line of instruction costs tokens on every run. The best libraries are the ones that get out of the way.<\/li>\n\n\n\n
                            • Skills are disposable<\/strong>. Evals are the truth \u2014 until they saturate. Plan to retire skills; plan to rebuild evals. Both are acts of maintenance, not failure.<\/li>\n\n\n\n
                            • No proof \u2192 no skill<\/strong>. Candidates are inert. Promotion is human. Injection is trigger-matched.<\/li>\n\n\n\n
                            • Verification beats instruction<\/strong>. Give agents ways to check their work; that outlasts any prompt.<\/li>\n\n\n\n
                            • Unhobble, don’t constrain<\/strong>. Measure the overhang. Remove the barriers. Keep the guardrails that catch catastrophes and delete the ones that catch nothing.<\/li>\n\n\n\n
                            • Cross-agent first<\/strong>. One skill, every runtime. Formats that lock you to one vendor are liabilities.<\/li>\n\n\n\n
                            • Low false-positive friction<\/strong>. A hook that cries wolf is a hook agents and humans learn to ignore. Ask only when it matters.<\/li>\n<\/ul>\n\n\n\n
                              \n\n\n\n

                              Technical Architecture<\/strong><\/h3>\n\n\n\n
                                \n
                              • Skill Store<\/strong> \u2014 versioned, content-addressed, semver-managed SKILL.md artifacts with references\/scripts\/assets<\/li>\n\n\n\n
                              • Provenance Ledger<\/strong> \u2014 every skill traceable to its source run, diff, compaction segment, and eval results<\/li>\n\n\n\n
                              • Ablation Runner<\/strong> \u2014 with\/without eval orchestration per skill per model generation<\/li>\n\n\n\n
                              • Eval Orchestrator<\/strong> \u2014 suite versioning, saturation detection, harder-eval generation queue<\/li>\n\n\n\n
                              • Hook Runtime<\/strong> \u2014 pre\/mid\/post hooks, cross-agent protocol, ask-not-deny safety tiering, compiled-binary hooks<\/li>\n\n\n\n
                              • Workflow<\/strong> Engine<\/strong> \u2014 branching pipelines, error recovery, step-level traces, replay<\/li>\n\n\n\n
                              • Routine Scheduler<\/strong> \u2014 isolated cron-like runs with persistent memory and budget caps<\/li>\n\n\n\n
                              • Overhang Analyzer<\/strong> \u2014 with\/without behavior comparison and unhobbling recommendations<\/li>\n\n\n\n
                              • Governance & Audit<\/strong> \u2014 permissions, review gates, promotion\/demotion history, immutable audit log<\/li>\n\n\n\n
                              • Distribution Layer<\/strong> \u2014 MCP server, REST\/WebSocket API, plugin-marketplace packaging, per-runtime config translation<\/li>\n<\/ul>\n\n\n\n
                                \n\n\n\n

                                Getting Started<\/strong><\/h3>\n\n\n\n
                                  \n
                                1. Import<\/strong> \u2014 bring existing prompts, CLAUDE.md\/AGENTS.md content, and .reckon\/skills<\/code> candidates in as candidates<\/em><\/li>\n\n\n\n
                                2. Baseline<\/strong> \u2014 run your eval suites with the imported library as-is<\/li>\n\n\n\n
                                3. First ablation sweep<\/strong> \u2014 delete what shows zero delta; record the token savings<\/li>\n\n\n\n
                                4. Set lifecycle policy<\/strong> \u2014 validation cadence, TTL decay, promotion rules<\/li>\n\n\n\n
                                5. Connect runtimes<\/strong> \u2014 Claude Code, Cursor, Codex, and the rest via the standard and the MCP server<\/li>\n\n\n\n
                                6. Curate weekly<\/strong> \u2014 review the “What can you delete today?” board; promote what proves itself; retire what doesn’t<\/li>\n<\/ol>\n\n\n\n

                                  <\/p>\n","protected":false},"excerpt":{"rendered":"

                                  SkillForge is a centralized platform for managing production-grade agent capabilities: composable skills, multi-step workflows, verification hooks, and autonomous routines. Unlike static prompt repositories, SkillForge is built on ablation-first principles \u2014 every instruction must justify itself through evals, every skill has a shelf life, and the system continuously pushes you to delete rather than add. Inspired […]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-141","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/artenatech.com\/index.php\/wp-json\/wp\/v2\/pages\/141","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/artenatech.com\/index.php\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/artenatech.com\/index.php\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/artenatech.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/artenatech.com\/index.php\/wp-json\/wp\/v2\/comments?post=141"}],"version-history":[{"count":9,"href":"https:\/\/artenatech.com\/index.php\/wp-json\/wp\/v2\/pages\/141\/revisions"}],"predecessor-version":[{"id":401,"href":"https:\/\/artenatech.com\/index.php\/wp-json\/wp\/v2\/pages\/141\/revisions\/401"}],"wp:attachment":[{"href":"https:\/\/artenatech.com\/index.php\/wp-json\/wp\/v2\/media?parent=141"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}