Generate text¶

Topics: clients, generation, requests, text

Choose a text model currently reported by AI Horde and decide the maximum output and context lengths before submitting the request. The simple client polls the text status endpoint and returns typed generations.

def text_request(model: str, api_key: str = ANON_API_KEY) -> TextGenerateAsyncRequest:
    """Build one bounded text request for a currently available model."""
    return TextGenerateAsyncRequest(
        apikey=api_key,
        prompt="Continue in two sentences: The observatory door opened",
        models=[model],
        params=ModelGenerationInputKobold(max_length=80, max_context_length=1024, n=1),
    )


def generate_text(model: str, api_key: str = ANON_API_KEY) -> str:
    """Return the first completed text generation."""
    client = AIHordeAPISimpleClient()
    status, _request_id = client.text_generate_request(text_request(model, api_key))
    if not status.generations or status.generations[0].text is None:
        raise RuntimeError("The request completed without text")
    return status.generations[0].text

Call generate_text(model="CURRENT_MODEL_NAME") and verify that the returned string is non-empty. A model name can become unavailable as workers enter or leave the network; query the model-status endpoint through the generated API reference when selecting dynamically.

Reducing max_length reverses an unexpectedly expensive or slow configuration. A submitted request cannot be edited; cancel it through the manual client or let the simple client's cleanup run before submitting a replacement.