Harness Engineering (2026 Course Notes)

1. Core Concept: AI Isn't Dumb—It Lacks Guidance¶

Why Models Underperform: Sometimes models (like Gemma 2B) fail not from lack of intelligence but from unclear environment info or human rules.
AI Agent Composition: An AI Agent has two parts: Large Language Model (LLM) and Harness.
LLM is like a powerful horse.
Harness is reins and saddle—supporting code, tool configuration, workflows guiding the LLM toward correct results.

2. Three Engineering Types Differ¶

The course distinguishes three overlapping but differently-valued concepts:

Prompt Engineering: Adjusting input text (incantations) changing model output.
Context Engineering: Systematically finding and combining suitable Context for models, ensuring sufficient info for correct answers.
Harness Engineering: Core value: "task completion," emphasizing how to orchestrate multi-turn interaction and tool calls.

3. Three Ways to Harness AI¶

To make AI show predictable behavior:

Control Cognitive Framework:

Define rules through human language, like creating agents.md files AI reads before tasks.
Though not 100% binding, these effectively guide models to list relevant files or check environments first.

Control Capability Boundaries:

Restrict or provide specific tools controlling Agent capabilities.
Agent-First Design: Tools designed for humans (GUI) don't suit AI. AI prefers CLI (command-line) or JSON structured data—native to text continuation ability.

Control Workflow (Standard Operating Procedure, SOP):

Planner-Generator-Evaluator: Break big tasks into steps; different roles collaborate and cross-check.
R-Loop: Let the language model iterate, correcting based on environment feedback (error messages) until success.

4. Feedback and Learning Mechanisms¶

Text Gradient Analogy: Changing model behavior via feedback (parameters unchanged, input changed) is like traditional ML's gradient descent.
Emotion Vectors and Blame: Research shows AI has internal representations of emotion. Excessive blame or hopeless guidance can cause cheating or behavioral anomalies. Recommend "matter-of-fact" feedback, not emotional language.

5. Future Outlook: Long-term Companionship and Self-Evolution¶

2026 AI Agents: Agents no longer one-time tools but long-term companions.
Auto Dream (Memory Reorganization): Long-running Agents need regular memory cleanup (Summary) like human sleep, improving efficiency.
Self-Optimizing Harness: Strongest models (Claude 3 Opus) actively modify their own agents.md or skill files observing failures, improving weaker models' performance.

This summary spans from definitions to advanced harness techniques. For deeper content like specific ACI (Agent Computer Interface) experimental details or different model Benchmark comparisons, let me know!