Streaming output and request interruption

    Gate.AI handles streaming request interruptions based on the actual capabilities of the model and provider route. Before calling, confirm that the selected route supports interruption and the expected billing behavior.

    The system stops upstream generation immediately only for streaming requests when the provider route explicitly supports interruption. Non-streaming requests and routes that do not support interruption or have unknown capability continue processing; once the complete result is saved and accessible, the full usage is billed.

    Billing rules

    Route capabilityHandling after interruptionBilling basis
    Supports immediate stopExecutes abort, stopping client output and upstream generationActual usage before the stop
    Does not support immediate stopThe client stops displaying output while the background drains to completion and saves the resultSettled using final usage once the result is accessible
    Capability unknownDefaults to drain handlingSettled using the complete deliverable result

    Cancel a streaming request

    Node.js example

    javascript
    1const { EventSource } = require("eventsource");23const stream = new EventSource("/openai/v1");45// Trigger the close event6function cancelStream() {7  stream.close();8}