Decide Whether the User Should Wait

The button says Generate and then nothing happens for twenty seconds. Some people click again. Some leave. The model often answers in four seconds and sometimes not at all. The interface promised a single wait for a single answer. The dependency does not keep that promise.

Decide from the user’s next action. If they cannot proceed without the result, and the result is usually fast, a wait with a visible limit can be honest. If they can keep working, or the result is a write that must not be duplicated, accept the request and show a status. Move Slow Work Out of the HTTP Request is the server half of that choice. Event-Driven AI Workflows is how a long chain survives. Neither choice should be an accident of whichever HTTP client timeout was copied from a sample.

A simple split

The cap has to be a number you chose for this screen, not the provider’s maximum. When the cap hits, the UI needs a sentence and a next step: edit by hand, try later, or leave it queued. A spinner with no end is the same as a crash that takes longer.

Illustrative scenario

This scenario is illustrative. A support tool suggests a category while the agent reads the ticket. The suggestion waits at most a short, fixed budget. If it misses the budget, the dropdown stays manual and the suggestion is marked skipped. Nobody retries three times inside the keypress. Separately, a nightly summary of tickets is a job. The agent does not wait for it. They open it in the morning, and a failed night is a status on the report, not a loading state from yesterday. The two calls use the same model vendor and opposite waiting decisions, because the user’s need is different.

Checklist

• Name the decision the user is blocked on, if any.
• Set a wait budget for synchronous calls and a fallback when it is missed.
• Move writes and batches to jobs with status.
• Make a second click safe. It must not start a second write.
• Show failure as failure. Do not leave a spinner on a dead request.
• Measure how often the wait budget is hit. If it is often, the call is not a wait. It is a job.

Questions that usually come next

Should we stream tokens to feel faster?

Streaming shows progress for text the user is reading as it arrives. It does not make a write safer, and it still needs an end state when the stream fails halfway. Use it for drafts the user is watching. Do not use it to avoid a status for a job.

What if the business wants every answer immediately?

Then the budget and the fallback are the product conversation. Immediate is a target you measure. A blocked UI with no cap is not immediate. It is indefinite. Ship the cap, show how often you miss it, and move the misses to a job or a narrower prompt.