Stream Cancellation

How aborting a request actually behaves - and why non-streaming requests can never be cancelled, no matter what timeout you set.

The one-line rule

Only streaming requests support cancellation. x-stainless-timeout and client-side aborts are respected for streaming requests only. Non-streaming requests ignore both and always run to completion.

Streaming cancellation

1. Abort the connection yourself

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://api.aivene.com/v1',
  apiKey: process.env.AIVENE_API_KEY
});

const controller = new AbortController();

const stream = await client.chat.completions.create(
  {
    model: 'gpt-5.5',
    messages: [{ role: 'user', content: 'Write a long story.' }],
    stream: true
  },
  { signal: controller.signal }
);

for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content;
  if (content) process.stdout.write(content);

  // Bail out as soon as you have what you need.
  if (shouldStop(content)) {
    controller.abort();
    break;
  }
}

2. Let your client's own timeout fire

const client = new OpenAI({
  baseURL: 'https://api.aivene.com/v1',
  apiKey: process.env.AIVENE_API_KEY,
  timeout: 15 * 1000 // 15 seconds
});

// Automatically sends `x-stainless-timeout: 15`. If the stream is still
// running 15 seconds in, we cut it at that point and bill only what
// streamed so far - no manual AbortController needed.
const stream = await client.chat.completions.create({
  model: 'gpt-5.5',
  messages: [{ role: 'user', content: 'Write a long story.' }],
  stream: true
});

Either way, the outcome is the same:

Non-streaming requests

Non-streaming requests cannot be cancelled by you:

  • x-stainless-timeout is ignored.
  • A client-side disconnect is ignored. The request keeps running regardless.

The same internal idle timeout that protects streaming requests still applies, so a provider that goes silent is cut either way and you get a timeout error back.

If you need a hard cost or length ceiling

Cancellation doesn't exist on non-streaming requests at all, so if you actually want to bound a request regardless of stream mode, use:

  • max_completion_tokens - caps output length server-side, for both streaming and non-streaming requests.
  • A stop sequence - ends generation as soon as the model emits it.

Need incremental output on a slow model?

If a model is slow enough that you're relying on a timeout to bail out early, switch that request to stream: true. You'll get usable partial output immediately instead of an all-or-nothing wait, and cancellation becomes available instead of guaranteed unavailable.