The DN Alpha Thesis: cheap inference can hide expensive work

A team buys a workstation, downloads a model and sees no charge for each prompt. That makes the inference layer feel free. The agent still consumes electricity, storage, administrator time and scarce machine capacity. If its outputs need more correction, the review bill can exceed the inference saving.

The reverse also happens. A cloud invoice looks expensive beside a local token estimate, yet the cloud workflow may finish more of the same tasks within the deadline. Neither situation establishes a general winner. The economic unit is an accepted task, and the deployment decision also depends on controls and capacity.

DN proposes making the denominator explicit. Count failed attempts in cost, then divide by accepted tasks. Keep quality and authority failures visible rather than blending them into a single score. An unauthorized action cannot be offset by a cheaper completion elsewhere.

Three deployment routes, three complete boundaries

Local inference describes where model computation runs. It does not describe every service the agent calls. A locally running model connected to a remote search tool, embedding API or CRM has a broader data boundary than the model process alone.

For a hybrid design, redaction should be tested against the actual task. Removing a customer identifier may preserve a summarization task while breaking a workflow that must retrieve the right record. Record both leakage checks and task failures.

What the primary evidence establishes

Ollama’s FAQ documents disabling cloud features through configuration or an environment variable. It also describes the default loopback binding and options for exposing the server. DN treats this as evidence of available controls, not an audit of a reader’s installation.

OpenAI’s API documentation distinguishes abuse-monitoring logs from application state and describes approval-based retention controls. Verify the endpoint and features you intend to use. A headline privacy claim is insufficient to describe an entire configured workflow.

These sources do not compare agent performance or ownership cost. The next step is a matched operational test, with your own task distribution, prices and acceptance rules.

DN’s task-by-task deployment register

The following register is a proposed test design. Its rows contain no measured local or cloud performance.

Read the example correctly

The illustrative local route costs 500 units for 800 accepted tasks, or 0.625 per accepted task. The cloud route costs 600 for 900, or approximately 0.667. Local appears cheaper per accepted outcome, but accepts 80% of attempted tasks against cloud’s 90%. That quality difference requires investigation. The calculator does not approve deployment.

A lower unit cost can be commercially useful when the rejected tasks are harmless and an economical fallback exists. Include that fallback in the complete route and retest. If the failures include missed deadlines or unauthorized actions, average cost alone cannot justify the setup.

Build the cost ledger before buying hardware

For an ownership view, allocate net hardware cost over its useful life: purchase and required upgrades, less expected residual value, divided by the months of use. Apply the share attributable to this workflow. Add power, maintenance, storage and recovery. Disclose assumptions instead of hiding them inside a token rate.

For a near-term cash decision on an existing machine, report incremental spending separately. A sunk purchase is different from a new purchase. An already-owned machine can still carry an opportunity cost if agent work prevents other valuable use.

Use actual cloud invoices over the evaluation period. Include retries, failed tasks, tool charges and auxiliary model calls. Allocate shared subscriptions explicitly. Human review cost equals recorded hours multiplied by a disclosed labor rate. Avoid counting the same review work again inside an overhead allocation.

Run low-, expected- and high-volume scenarios. Fixed local costs spread over more tasks as utilization rises, but capacity has limits. A linear projection does not prove that one machine can meet the proposed throughput. Measure queue time and end-to-end completion at the target concurrency.

A fair pilot in six steps

1. Define a representative task set, difficulty groups, deadlines and an acceptance rubric before testing.
2. Freeze model versions, runtime settings, tools, retry limits and permissions. Record hardware and cloud service configuration.
3. Run matched tasks on each complete route. Rotate execution order and record warm-start and cold-start behavior separately.
4. Log cost, reviewer time, acceptance, failures and end-to-end latency. Report median and tail latency with sample size.
5. Audit outbound traffic and data handling across tools, logs, retrieval and fallback. Test recovery and permission boundaries.
6. Repeat disputed outcomes, investigate quality differences and choose by task class. Schedule a retest after material changes.

Report paired outcomes, not only aggregate acceptance. Two routes can achieve the same rate while failing different tasks. A small pilot can reveal obvious problems but does not establish reliable rare-failure rates.

Methodology, limitations and falsification

DN-LOC v1.0: total period cost equals infrastructure or service cost plus human review and other workflow costs. Unit cost equals that total divided by accepted tasks. Attempted task counts must match; acceptance is accepted divided by attempted. Costs include failed work. With zero accepted tasks, unit cost is undefined.

Scope: this calculator is a two-route diagnostic. To assess a hybrid design, enter it as one route and compare it with a complete alternative. It does not estimate power use, depreciation, taxes, financing, throughput or security automatically. It uses one currency and one accounting period.

Limitations: reader inputs can be wrong, shared overhead allocation is judgmental, and acceptance rates do not capture every consequence of failure. No live vendor comparisons, hardware recommendations or measured savings are published in this edition.

Falsification: an asserted local saving fails if complete cost per accepted task is higher on the matched workload, if required quality or deadlines fail, or if the claimed data boundary is violated. A cloud superiority claim must face the same conditions. Retest whenever the model, tool chain, workload or prices change.

What to do next

Choose one recurring task and collect a complete cost and outcome ledger. Run a bounded pilot before purchasing capacity or moving sensitive workflows. The useful purchasing brief includes accepted outcomes, review hours, peak load and verified controls. This article includes no affiliate recommendations or paid placements.

Frequently asked questions