Caisey Blog

Caisey users · July 13, 2026

How Caisey Works

An intro to Caisey, how it works, our partners, an overview of our decision making framework.

An inside look at Caisey’s architecture, information flow, safety controls, and human-in-the-loop decision framework.

Hey everyone, J here from the Caisey development team.

AI products can be difficult to evaluate from the outside. You send a request, something happens behind the interface, and an answer appears. But when an AI system can troubleshoot and make changes to real computers, “something happens” is not a sufficient explanation.

We want Caisey’s users to understand how their tools work. This article walks through the flow of information inside Caisey, the partners involved, and the principles we use when deciding what Caisey should and should not be allowed to do.

The basic architecture

Caisey has three primary components:

  1. The technician dashboard at helpforsupport.com
  2. A routing and control layer hosted on Cloudflare
  3. The Caisey runtime installed on the endpoint being supported

The dashboard is where the technician starts and supervises the work. The routing layer coordinates communication between the dashboard, the endpoint, and our inference provider. The endpoint runtime gives Caisey controlled access to the machine’s local troubleshooting tools.

That endpoint presence is important. Caisey is not limited to interpreting a ticket description and offering generic suggestions. It can inspect the actual machine, test a hypothesis, request permission to run a command, review the result, and continue troubleshooting with better information.

From technician request to endpoint

Every Caisey session begins with a request from a technician.

The technician sends that request through the Caisey dashboard. It then travels to our routing layer, which is hosted by Cloudflare.

Caisey request flow

Before beginning the session, the routing layer checks whether the Caisey runtime on the selected endpoint is online and ready to receive work.

Endpoint connection check

If the endpoint responds successfully, the dashboard displays the connection status. The routing layer then passes the technician’s request, along with the controlled context needed for the session, to the Caisey runtime.

Connected Caisey session

The runtime sends an inference request back through the routing layer to our US-based inference provider. We currently use Fireworks to run open-weight models. Using open-weight models gives us greater flexibility over model selection, security practices, privacy, and data governance as the AI landscape changes.

Inference request flow

As the model processes the request, its output streams back through the routing layer and appears in the technician dashboard.

Streaming model output

At this point, Caisey has not simply generated an answer. It has begun a troubleshooting process.

How Caisey uses tools

Most computer problems cannot be diagnosed reliably from a short description. A technician needs to inspect the machine, gather evidence, test possible causes, and verify the result.

Caisey works the same way.

For example, it might request a tool call resembling this:

<Tool name="powershell">
  <command>Get-Service -Name Spooler</command>
</Tool>

This command checks the status of the Windows Print Spooler service. It gathers information without changing the machine.

When Caisey requests a tool, the routing layer checks whether that exact tool or command has already been approved for the organization.

Tool approval check

If the command has not been approved, two things happen:

  • The technician receives a permission request.
  • The proposed action is sent to a separate model for independent risk analysis.

Why we use an independent risk evaluator

The model performing the troubleshooting does not evaluate its own proposed actions in isolation.

Instead, non-approved tool requests are examined by a separate model with a different architecture and a tightly controlled context. That evaluator does not need the complete troubleshooting conversation. It receives the information required to assess the proposed action without inheriting all the potentially untrusted material present in the main session.

This separation is designed to reduce correlated failures and limit the influence of prompt injection. It does not make AI infallible, and we do not treat it as a substitute for technician judgment. It is an additional control that gives the technician better information before making a decision.

The evaluator assigns a general risk level, such as low, medium, or high, and explains what the command appears to do and why it may be risky. Caisey also uses deterministic triggers for commands or tool categories associated with destructive or consequential behavior.

Fireworks has written about some of its own infrastructure-level prompt-injection protections in its article on safe tokenization.

Independent command analysisTool-risk result

Once the analysis is complete, Caisey displays the command, its purpose, and its estimated risk in the technician interface.

Technician permission promptPermission decision details

The technician can deny the request, approve it for the current situation, or, where supported, approve the exact command for the organization.

If an organizational approval is granted, the routing layer records that approval and passes the tool request to the Caisey runtime on the endpoint.

Approved command execution

The runtime executes the command and returns its output through the routing layer. That result is delivered both to the model and to the technician dashboard.

Command result returned

Caisey can then use the new evidence to revise its hypothesis, request another tool, ask the technician a question, or conclude that the work is complete.

That is Caisey’s basic event loop:

  1. Understand the request.
  2. Gather evidence.
  3. Form a hypothesis.
  4. Request permission where required.
  5. Run the approved action.
  6. Inspect the result.
  7. Continue or stop.

In other words: diagnose, act, verify, and repeat.

How we make product decisions

Human oversight has been one of Caisey’s north stars from the beginning. That comes from both our experience working with AI and our background in managed IT services.

Modern models are remarkably capable, but capability is not the same as judgment. A model may misunderstand context, follow untrusted instructions, choose an unnecessarily disruptive approach, or sound more confident than the evidence warrants.

We therefore use four principles when designing Caisey:

  1. When a thoughtful technician would pause, Caisey should pause.
  2. The interface should make the safest reasonable decision easy to understand.
  3. Permissions should be specific, visible, and easy to revoke.
  4. When the system is uncertain, the decision belongs to a human.

Some expressions of these principles are deliberately subtle. For example, denying a tool request requires less effort than approving one. Technicians can review and remove previously granted permissions, and our current approvals are exact rather than broad wildcard permissions.

Wildcard permissions may eventually have legitimate uses, but convenience alone is not a sufficient reason to introduce them. We will add broader authority only when we are confident it can be made understandable, controllable, and safe.

We also maintain a mildly adversarial relationship with the models we use. Models are powerful components, not trusted decision-makers. Our job is to provide boundaries, independent checks, observable behavior, and meaningful human control around them.

The model can investigate and recommend. The technician remains responsible for consequential decisions.

Supporting technicians, not replacing them

Caisey is not built around the assumption that humans are the slow part of technical support.

Experienced technicians bring judgment, customer knowledge, pattern recognition, accountability, and an understanding of consequences that models do not possess. The problem is that technicians spend an enormous amount of time gathering routine information, running repetitive checks, and moving between disconnected systems.

Caisey is designed to compress that work.

It can help a junior technician follow a more disciplined diagnostic process. It can help a senior technician investigate several possibilities faster. It can preserve the commands, results, permissions, and reasoning attached to a support session.

The goal is not to remove the technician. The goal is to give the technician better leverage.

Our current partners

Caisey currently works with a small number of infrastructure partners:

  • Cloudflare provides compute, storage, databases, routing infrastructure, and email services.
  • Clerk provides authentication services.
  • Fireworks provides AI inference.

These providers may change as the product and the AI ecosystem evolve. When they do, we intend to be clear about what changed and why.

Our relationship is with our customers. You pay us to provide a useful service. Your information is not an advertising product or a bargaining chip.

Dollars, not data. Customers, not deals.

The short version

Caisey places an AI troubleshooting runtime on the endpoint, coordinates it through a Cloudflare-hosted control layer, and lets technicians supervise the work from a browser.

It gathers evidence from the real machine, requests tools when it needs them, applies independent risk analysis to non-approved actions, asks for human authorization, and returns the results to both the technician and the model.

Then it repeats the process until the problem is resolved or a human decision is needed.

That is how Caisey turns AI from a suggestion engine into a controlled troubleshooting system while keeping the technician in charge.

If you have questions about how Caisey works, I’d genuinely love to hear them.

Email me at [j@zelek.tech](mailto:j@zelek.tech).

Caisey
Caisey
loading...