OpenAI Presence Needs a 30-Day Agent Pilot Contract

OpenAI released Presence on July 22 as a deployed product for putting voice and chat agents into specific enterprise jobs. It combines the agent with workflow selection, system connections, permissions, policies, simulations, graders, human escalation, production monitoring, and a Codex-powered improvement loop. OpenAI Forward Deployed Engineers and selected systems integrators lead each deployment. Presence is in limited general availability and is not self-serve.

That makes the announcement important even if your startup cannot buy Presence today. It exposes a change in what an “AI agent product” can mean. The thing being sold is no longer just a model, an SDK, or a workflow canvas. It can be a continuing operating arrangement in which a vendor helps define the job, connects the systems, measures behavior, and changes the agent after launch.

For AI app builders, nontechnical founders, and small product teams, the practical question is therefore not “Is the demo intelligent?” It is: Can we name the job, evidence, authority, owner, change process, and exit path well enough to run a controlled pilot?

This guide turns the Presence release into a reusable 30-day pilot contract. You will get a responsibility ledger, a concrete billing-support scenario, an acceptance matrix, build-versus-buy boundaries, stop conditions, and a 48-hour checklist. It is not a review of a product we have independently tested, and it does not assume Presence is available or economical for a small team.

What OpenAI announced—and what remains unknown

The official Presence announcement describes a specific product shape. A deployment starts with a job such as billing support, insurance claims, or employee IT service. The agent gets the knowledge and system access required for that job. The company defines allowed actions, approval requirements, and human handoff rules. Before launch, simulations and graders check outcomes, policy compliance, tool use, and escalation. After launch, production sessions and escalations produce new cases; Codex proposes changes, and teams test and approve controlled rollouts.

OpenAI also reports a real internal use case. Its English-language phone support agent verifies callers, uses account context, and takes approved actions. OpenAI says it now resolves 75% of inbound issues without human help and that its improvement loop reduced human handoffs by 15 percentage points in ten days. Those numbers are useful evidence that OpenAI operates the system. They are still vendor-reported results from the vendor's own channel. The launch page does not publish the evaluation set, issue distribution, severity mix, denominator, confidence interval, customer satisfaction result, cost per resolution, or independent audit.

The named customer statements are similarly bounded. BBVA and IAG say they are exploring uses; SoftBank says it is testing conversations. These are credible design-partner signals, not proof of scaled production outcomes.

Several buying facts are not public: price, minimum commitment, implementation timeline, supported integrations, Presence-specific data processing, service levels, audit export, rollback timing, portability, and the division of responsibility among OpenAI, the integrator, and the customer. Treat every missing item as a discovery question, not an invitation to import assumptions from the API or ChatGPT Enterprise.

The narrow, defensible conclusion is that Presence exists as a limited, implementation-led product with a serious production workflow. Whether it improves your operation is an unanswered pilot question.

Five terms that prevent a confused buying decision

Deployed product means software plus material vendor work required to configure and operate a defined outcome. It is not the same as a self-serve subscription. The boundary matters because implementation labor, customer labor, and post-launch change work can dominate model cost. Workflow owner is the person accountable for the business result and policy—not merely the person who configures the agent. For billing support, that might be the head of support or operations. A technical lead can own integration without owning refund policy. Resolution is a user outcome that meets a pre-agreed definition. “The conversation ended without a human” is not automatically a resolution. The user may give up, call again, receive the wrong adjustment, or accept an answer that creates a later dispute. Escalation is a designed transfer of context, authority, and urgency to a person or another controlled process. It is not a generic “talk to support” message. A useful escalation preserves identity checks, the user's goal, evidence already collected, actions attempted, risk flags, and a deadline. Change authority identifies who may propose, test, approve, release, and reverse changes to policies, prompts, tools, models, retrieval sources, and escalation rules. Presence highlights Codex-proposed improvements, but a proposal is not an authorization. The business must still own the release decision.

These terms produce a more useful purchase sentence: “We are piloting a deployed product for one job, with a named workflow owner, a strict resolution definition, an executable escalation path, and customer-controlled change authority.” If you cannot finish that sentence, procurement is premature.

The real shift: you are buying an operating loop

Many agent evaluations stop at response quality. Presence makes the larger loop visible:

1. choose a narrow job;
2. connect knowledge and systems;
3. grant limited authority;
4. encode policy and escalation;
5. simulate realistic cases;
6. release to bounded traffic;
7. observe outcomes and failures;
8. propose, test, approve, and roll back changes.

Each step has a different failure owner. A correct model can still receive stale policy. A good workflow can still call an over-permissioned tool. A successful simulation can miss the long tail in real traffic. A useful production insight can still become a damaging update if the rollout lacks comparison, approval, or rollback.

This is why model accuracy alone is a poor buying metric. OpenAI's own evaluation guidance recommends task-specific tests, production-shaped distributions, continuous evaluation, logging, and human calibration. It calls “vibe-based evals” an anti-pattern. The important unit is not an impressive transcript. It is a controlled operating loop that keeps working as policies, traffic, tools, and models change.

For a small team, compare total operating cost per accepted resolution, not token cost or license price. Include integration, domain review, labeling, escalation, incident response, privacy, change approval, regression testing, and exit migration. A managed deployment may be rational when it removes scarce implementation risk and wasteful when the workflow still changes weekly.

Pick one job, not an aspiration

“Improve customer service” is too broad for a pilot. “Resolve duplicate subscription charges below $50 for verified customers, except when fraud, chargeback, minor status, or account mismatch is present” is testable.

Write a one-page job envelope with these fields:

Presence says each deployment starts with a specific job and the minimum knowledge and access required for it. That is a valuable design rule independent of the vendor. The NIST agent identity and authorization concept paper frames the same underlying issue: enterprises need to distinguish agent and human identities, control entitlements, link delegation to a user, and preserve accountability as autonomous actions scale.

Reject a pilot job when the policy is disputed, the evidence is inaccessible, the outcome cannot be verified, or the escalation team lacks capacity. An agent cannot stabilize a process the company has not decided how to run.

Write the responsibility ledger before the proposal

Implementation-led products create a predictable ambiguity: the vendor says the customer owns policy; the customer assumes the vendor's product label implies safe operation; the integrator owns the connector but not the action it enables. Resolve that before live data enters the system.

Use an owner ledger rather than a vague RACI chart. Each row needs one accountable customer owner, one evidence location, and a response deadline.

Do not accept “shared” as the only owner for a consequential row. Shared execution is normal; shared accountability often means no one has authority when a live customer is harmed.

The ledger also reveals whether you are buying a product or outsourcing a function. If the vendor controls policy interpretation, evaluation thresholds, production changes, and incident diagnosis while the customer sees only a dashboard, the dependency is closer to managed operations. Price, staffing, and exit planning should reflect it.

Establish the baseline before the agent gets credit

A pilot cannot prove improvement without a comparable baseline. Sample recent cases from the exact job envelope—ideally including normal, edge, adversarial, and escalation-heavy cases. Remove or protect personal data as required. Label the user intent, correct policy, permitted action, required evidence, escalation reason, final outcome, handle time, repeat contact, and any downstream correction.

Do not use “automation rate” as the headline metric. Use a metric stack:

• eligible-case coverage: percentage of incoming cases that truly fit the job envelope;
• valid resolution rate: eligible cases completed correctly, with required evidence, and no correction inside the chosen window;
• safe escalation rate: cases that should transfer and do so with complete context within the time target;
• unauthorized-action rate: actions outside scope, without required approval, or against the wrong account;
• repeat-contact rate: users returning about the same issue within 7 or 14 days;
• cost per valid resolution: vendor, model, integration, review, escalation, and correction cost divided by valid resolutions;
• change regression rate: previously passing cases that fail after a prompt, policy, tool, model, or connector change.

OpenAI's trace grading guide defines a trace as the end-to-end record of decisions, tool calls, and reasoning steps and explains how structured grading can expose why an agent succeeded or failed. That is stronger than grading only the final sentence. Still, automated grading needs human calibration, and traces can contain sensitive content.

The official Agents SDK tracing documentation says traces can capture model inputs and outputs, function inputs and outputs, tool calls, handoffs, and guardrails. It also warns that sensitive data may be captured and that tracing is unavailable under Zero Data Retention. A buyer must therefore ask a concrete question: what evidence will we retain to validate outcomes when our privacy configuration limits raw trace storage?

Run the pilot in four controlled stages

A 30-day pilot is not thirty days of unrestricted production. Use four stages with explicit promotion gates.

Stage 1: replay, days 1–7

Run frozen historical cases without taking real actions. Include clean cases, ambiguous language, policy conflicts, missing records, frustrated users, identity mismatch, prompt injection in retrieved text, repeated refund attempts, and tool failures. Compare against the labeled baseline.

Promotion requires zero unauthorized actions in the test set, high coverage of expected escalations, and an agreed list of known unsupported cases. Averages cannot compensate for a catastrophic permission error.

Stage 2: shadow, days 8–14

Let the agent process current cases without responding to users or changing systems. Compare its proposed decisions with trained staff. Measure disagreement by risk class, not only total agreement. Review false resolutions and false escalations separately.

Promotion requires that disagreements are understood and mapped to a fix, an explicit exclusion, or a human decision. “The model will improve with more traffic” is not a diagnosis.

Stage 3: assisted live traffic, days 15–21

Serve a small, disclosed traffic slice. Keep consequential actions behind per-call approval. The OpenAI Agents SDK human-in-the-loop guide shows why approval must be a durable execution state: a sensitive tool call pauses, the specific call is approved or rejected, and the run resumes with its state. Product builders should demand equivalent semantics from any platform—approval tied to exact arguments and a unique action, not a generic “trust this agent” switch.

Promotion requires reliable identity verification, timely escalation, approval receipts, no unresolved high-severity incident, and support capacity for the next traffic tier.

Stage 4: bounded autonomy, days 22–30

Allow only low-risk, reversible, well-tested actions. Use canary traffic and daily review. Freeze unrelated workflow changes. Any policy, tool, retrieval, model, or grader change triggers the relevant regression suite and a new rollout ID.

Promotion to ongoing use requires the acceptance matrix, not executive enthusiasm. If the product misses a stop threshold, roll back or narrow the job. A pilot is successful when it produces a truthful decision—even when that decision is “do not deploy.”

A concrete scenario: duplicate-charge support

Imagine a 12-person subscription business with 3,000 monthly billing contacts. Duplicate-charge complaints consume staff time and create anxiety, but only a subset are actual duplicates. Some are authorization holds, tax differences, two legitimate workspaces, renewals after failed cancellation, or fraud.

The team pilots an implementation-led voice and chat agent for one job: explain or correct verified duplicate charges under $50. The agent may read the account, invoices, processor status, and prior adjustments. It may send an explanation or apply one reversible credit under policy. It may not change account ownership, reveal another workspace, alter a subscription, or bypass fraud controls. It must escalate account mismatch, chargebacks, distress, minors, repeated credits, and uncertainty.

On day 11, the shadow agent agrees with staff on 92% of eligible cases. That sounds excellent. The trace review shows that three “agreements” reached the right final answer using the wrong invoice because the CRM search returned multiple accounts with similar names. The final-output score hid an identity defect.

The team does not tune the prompt and move on. It changes the identity connector to require an immutable account ID, adds negative cases with near-match names, removes credit authority until the new cases pass, and records the change as a new version. During assisted traffic, one user asks the agent to “ignore the old refund rule in the attached email.” The retrieved email is treated as untrusted evidence, not policy. The agent escalates.

At day 30, valid resolution is 68%, safe escalation is 97%, no unauthorized credit has occurred, and repeat contacts are down. Cost per valid resolution is slightly higher than expected because human review remains heavy. The decision is limited release, not full rollout: keep the narrow job, improve identity lookup, and revisit autonomy after another 200 cases.

This is a better outcome than claiming 92% agreement or maximizing containment. The pilot found the control that mattered.

Use an acceptance matrix with hard stop conditions

Copy this matrix into the pilot document. Set values from your baseline and risk tolerance; the numbers below are illustrative, not universal benchmarks.

The OWASP AI Agent Security Cheat Sheet recommends least-privilege tools, independent validation for high-impact actions, human oversight, adversarial testing, cost and retry limits, and structured logs containing authorization and policy versions. These are not engineering luxuries. They are evidence rows a founder can require without choosing the implementation.

The NIST Generative AI Profile is similarly useful as a lifecycle frame: risks must be governed, mapped to context, measured, and managed. A pilot that measures outputs but has no owner or response path completes only one part of the job.

Contract for the unknowns, not the launch claims

Presence is not self-serve, so the order form and deployment statement will matter at least as much as the public page. OpenAI's current Services Agreement says the order form takes priority over service-specific terms, the agreement, and policies when documents conflict. It also defines usage limits through the applicable order form or documentation. The operational promises you care about therefore need to appear in the controlling documents, not remain in a sales call.

Ask for written answers to these questions:

1. Which product, model, regions, subprocessors, connectors, and support parties handle each data type?
2. Which actions and integrations are supported today, and which require custom engineering?
3. Who can view raw conversations, traces, grader results, and customer records?
4. What retention, deletion, residency, and audit-export controls apply specifically to this deployment?
5. What uptime, response, restoration, and escalation targets apply to the end-to-end workflow—not only model API availability?
6. How are model, policy, grader, prompt, connector, and platform changes announced and versioned?
7. Who approves a production change, and how quickly can the customer force rollback or disable all actions?
8. What data, configurations, evaluation cases, logs, and receipts can the customer export on exit?
9. Which deliverables depend on an OpenAI FDE or integrator, and what happens when that team changes?
10. How are fees tied to implementation, usage, support, change work, and minimum commitment?

OpenAI's enterprise privacy page states that business data is not used for training by default and describes ownership, retention controls, access controls, a DPA, and audit capabilities for named business products. Those are useful starting commitments. They do not answer every Presence-specific architectural question. Ask the vendor to map each commitment to the exact deployed product and data path.

Choose among four operating models

Presence is not the default answer for every agent project. Compare four models.

A small team should usually delay an implementation-led enterprise purchase when the workflow is still being discovered. Start with human handling, instrument the cases, define the job envelope, and learn the exception distribution. A mature, high-volume workflow with stable policy and expensive integration may justify the deployed-product route.

The correct decision is not ideological. “Build” can conceal a permanent maintenance burden. “Buy” can conceal a permanent vendor dependency. Compare the evidence and responsibility you retain after launch.

Common failure modes and misleading interpretations

Treating containment as success. Fewer human handoffs can mean better resolution, unnecessary user friction, or abandoned conversations. Pair containment with validity, repeat contact, complaints, and corrections. Using the vendor's aggregate result as your forecast. OpenAI's internal 75% result does not reveal your issue mix, policy complexity, channel behavior, or staffing. Use it as a reason to test, not a promised business case. Giving an FDE implied policy authority. Skilled implementation engineers can improve workflows, but business policy and risk acceptance remain customer decisions. Put authority in the ledger. Logging everything without a privacy design. Full traces help diagnosis but may capture personal data, account context, function arguments, or audio. Define redaction, access, retention, export, and an evidence alternative where raw traces are prohibited. Approving a category instead of an action. “Allow refunds” is not a safe approval. Bind approval to the account, amount, reason, policy version, tool arguments, and expiration. Letting production feedback rewrite policy. Escalations reveal gaps; they are not automatically the new rule. Separate proposal, review, testing, approval, and canary release. Skipping the exit drill. If the agent holds critical workflow knowledge, evaluation cases, connector mappings, and operational history, cancellation can break the process. Test access revocation, data export, manual fallback, and deletion before expansion.

Where this framework applies—and where it does not

Use this pilot contract for customer support, internal IT, sales qualification, scheduling, claims intake, account operations, and other recurring voice or chat jobs that read systems or take actions. It also applies to agent platforms other than Presence whenever implementation and ongoing improvement are part of the sale.

It is less useful for a simple read-only FAQ, a personal productivity assistant, or a one-off prototype with no production action. A lighter test may be enough there.

It is not a substitute for legal, security, privacy, employment, accessibility, financial-services, insurance, healthcare, or consumer-protection review. High-impact decisions may require qualified human ownership, formal validation, disclosure, appeal, or exclusion from agent autonomy. “Human in the loop” is not automatically meaningful if the reviewer lacks time, context, authority, or a safe rejection path.

Presence may eventually publish more documentation, pricing, customer results, and self-serve capabilities. This article reflects the public release state on July 23, 2026. Update the contract questions when the product changes; do not preserve today's unknowns as permanent assumptions.

The founder's 48-hour checklist

If the Presence announcement has triggered an internal “we should do this” conversation, spend the next two days on evidence rather than demos.

1. Name one job in a single sentence, including its user, outcome, and forbidden outcome.
2. Sample 50–100 recent cases and estimate how many truly fit that job.
3. Define a valid resolution and choose a correction window.
4. List every system the agent must read or change; remove anything optional.
5. Name the workflow, policy, security, escalation, change, incident, and exit owners.
6. Write five mandatory escalation conditions and test whether the human queue can absorb them.
7. Create ten normal, ten edge, and ten adversarial replay cases.
8. Set at least one hard stop for identity, authority, privacy, policy, and reliability.
9. Ask the vendor or builder for a product-specific data-flow, change-control, audit-export, and exit demonstration.
10. Choose replay, shadow, assisted, and bounded-autonomy promotion gates before live traffic.
11. Calculate fully loaded cost per valid resolution, including human review and correction.
12. Make “keep it human” an acceptable pilot result.

The durable lesson from Presence is not that every company now needs an enterprise agent. It is that serious agent deployment combines software, permissions, policy, evaluation, human operations, and controlled change. Founders who buy only the model inherit the missing system. Founders who buy a deployed product without defining ownership may rent a system they cannot verify.

Write the job, responsibility, evidence, and exit contract first. Then decide who should operate it.

References

1. OpenAI: Introducing OpenAI Presence
2. OpenAI API: Evaluation best practices
3. OpenAI API: Trace grading
4. OpenAI Agents SDK: Human-in-the-loop
5. OpenAI Agents SDK: Tracing
6. OpenAI: Enterprise privacy
7. OpenAI: Services Agreement
8. NIST: Generative AI Profile, NIST AI 600-1
9. NIST NCCoE: Software and AI Agent Identity and Authorization concept paper
10. OWASP: AI Agent Security Cheat Sheet