Build a Chatbot From an LLM API: Python Tutorial

The core idea: a chatbot is a loop and a list

A language model does not remember anything between requests. Every call to POST /v1/chat/completions is independent. What feels like memory is your program sending the earlier messages again, each tagged with a role: system for standing instructions, user for what the person typed, assistant for earlier model replies.

So the whole architecture is small. Keep a list. Append the user message. Send the list. Append the reply. Repeat. Everything else in this tutorial, from streaming to tools, is a variation on that loop.

The API here is OpenAI-compatible, so you use the regular openai package. Set the base URL to https://api.abliterated.cc/v1, use your key, and set the model to uncensored. Get a key from the API key page; the $0.50 trial credit is enough for many hundreds of short test messages.

Step 1: a working console chatbot

Install the SDK with pip install openai and run this.

Copyfrom openai import OpenAI

client = OpenAI(
    base_url="https://api.abliterated.cc/v1",
    api_key="YOUR_KEY",
)

messages = [
    {"role": "system", "content": "You are a concise, friendly support assistant for a bike shop."}
]

while True:
    text = input("you: ")
    if text.strip() in {"quit", "exit"}:
        break
    messages.append({"role": "user", "content": text})
    resp = client.chat.completions.create(
        model="uncensored",
        messages=messages,
        temperature=0.7,
    )
    answer = resp.choices[0].message.content
    messages.append({"role": "assistant", "content": answer})
    print("bot:", answer)

That is a complete chatbot. The system message sets its job, the list carries the memory, and temperature controls how varied the replies are. Lower it toward 0.2 for factual support bots, raise it toward 1.0 for creative ones.

Write a system prompt that sets the job

The system message is the cheapest place to improve a chatbot. Keep it short and specific. Name the role, the audience, the length and tone of replies, and what to do when the bot does not know. A prompt such as "You answer questions for a bike shop. Reply in at most three sentences. If the question is about a repair price, say that a mechanic must inspect the bike" does more than a page of general virtues.

Two habits pay off. First, put rules in the form of actions, not attitudes: "ask one clarifying question when the request is ambiguous" instead of "be helpful". Second, test with hostile inputs, such as an empty message, a very long paste and a request that has nothing to do with your product. Fix the prompt based on what you see, then keep those inputs as a regression list.

Because the system message is resent each turn, every word in it is billed on every request. A 400-token prompt over 1,000 chats of 5 turns each adds 2,000,000 input tokens, or 2 x $0.15 = $0.30. That is small, but it shows why bloated prompts add up at volume.

Step 2: stream the reply

Waiting for a full answer feels slow in a chat window. Set stream: true and print pieces as they arrive. The stream uses server-sent events and ends with a usage block, so you can log token counts.

Copyfrom openai import OpenAI

client = OpenAI(base_url="https://api.abliterated.cc/v1", api_key="YOUR_KEY")
messages = [{"role": "user", "content": "Explain tire pressure in two sentences."}]

stream = client.chat.completions.create(
    model="uncensored",
    messages=messages,
    stream=True,
)
reply = ""
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        part = chunk.choices[0].delta.content
        reply += part
        print(part, end="", flush=True)
    if getattr(chunk, "usage", None):
        print("\ntokens:", chunk.usage.prompt_tokens, chunk.usage.completion_tokens)

Remember to append the assembled reply to your history, or the next turn will have no idea what the bot said.

Step 3: manage context before it grows

The window is 100,000 tokens, shared between prompt and completion. Most chatbots never get near it, but cost grows with history because you pay for the whole prompt on each turn. A simple policy keeps both in check:

• Always keep the system message.
• Keep the last 10 to 20 messages verbatim.
• When the list passes a size limit you choose, such as 8,000 estimated tokens, send the oldest block to the model with the instruction "summarize this conversation in under 150 words", and replace that block with the summary as a system message.

A rough estimate of one token per four English characters is enough for the trigger. The usage numbers in each response give you the exact figure after the fact, so you can calibrate the limit.

If you truncate by dropping the oldest messages, drop them in user and assistant pairs. A reply with no question, or a question with no reply, confuses the model more than you would expect.

Step 4: give the bot tools

Tool calling lets the model ask your code to do something, such as look up an order. You describe functions with tools, the model replies with a tool_calls entry, you run the function, and you send the result back as a role: "tool" message.

Copyimport json
from openai import OpenAI

client = OpenAI(base_url="https://api.abliterated.cc/v1", api_key="YOUR_KEY")

tools = [{
    "type": "function",
    "function": {
        "name": "get_order_status",
        "description": "Look up the status of an order by its id",
        "parameters": {
            "type": "object",
            "properties": {"order_id": {"type": "string"}},
            "required": ["order_id"],
        },
    },
}]

def get_order_status(order_id):
    return {"order_id": order_id, "status": "shipped"}

messages = [{"role": "user", "content": "Where is order A1042?"}]
resp = client.chat.completions.create(
    model="uncensored", messages=messages, tools=tools, tool_choice="auto"
)
msg = resp.choices[0].message

if msg.tool_calls:
    messages.append(msg)
    for call in msg.tool_calls:
        args = json.loads(call.function.arguments)
        result = get_order_status(**args)
        messages.append({
            "role": "tool",
            "tool_call_id": call.id,
            "content": json.dumps(result),
        })
    final = client.chat.completions.create(
        model="uncensored", messages=messages, tools=tools
    )
    print(final.choices[0].message.content)

Always validate the arguments before running anything with side effects. The model proposes a call; your code decides whether to execute it.

Step 5: structured output with JSON mode

When the next step in your program needs data rather than prose, such as sorting a message into a category, use JSON mode with response_format. Mention the word JSON and the expected keys in your prompt as well.

Copycurl https://api.abliterated.cc/v1/chat/completions \
  -H "Authorization: Bearer YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "uncensored",
    "response_format": {"type": "json_object"},
    "messages": [
      {"role": "system", "content": "Reply with JSON: {\"intent\": string, \"urgent\": boolean}"},
      {"role": "user", "content": "My brakes squeal and I ride tomorrow morning."}
    ]
  }'

Still wrap json.loads in a try block. If parsing fails, retry once with the error text appended; failed requests are not charged.

Errors, cost and deployment notes

Handle errors, retries and rate limits

Real users send requests at awkward moments, and networks fail. Wrap the call so one bad response does not crash the conversation.

Copyimport time
from openai import OpenAI, APIError, RateLimitError

client = OpenAI(base_url="https://api.abliterated.cc/v1", api_key="YOUR_KEY")

def chat(messages, tries=3):
    for attempt in range(tries):
        try:
            resp = client.chat.completions.create(
                model="uncensored", messages=messages, temperature=0.7
            )
            return resp.choices[0].message.content
        except RateLimitError:
            time.sleep(2 ** attempt)
        except APIError as exc:
            if attempt == tries - 1:
                raise
            time.sleep(1)
    return "Sorry, the assistant is busy. Please try again in a moment."

A full key allows 8 concurrent requests and 300 requests per minute; the trial key allows 2 and 60. If your bot serves many users at once, put requests in a queue with a worker limit below 8 instead of relying on retries. Because failed requests are not charged, retrying has no token cost. Keep the retry count small so a real outage fails fast.

Also log the X-Request-Id header with each conversation turn. When a reply looks wrong, the id lets you find the exact request later.

Cost and deployment

Price is $0.15 per 1M input tokens and $0.50 per 1M output tokens. Say a support chat uses 2,000 input tokens in total and 400 output tokens.

• Input: 0.002M x $0.15 = $0.0003
• Output: 0.0004M x $0.50 = $0.0002
• One chat: $0.0005, so 1,000 chats cost $0.50

Check the current rates and top-up bonuses on the pricing page. For production, keep the key on your server, never in browser code, and watch the X-Balance-USD response header to alert yourself before credit runs out. A full key allows 8 concurrent requests and 300 requests per minute, so queue bursts instead of firing them all at once. The docs list every supported parameter.

More guides

• Best Abliterated LLM for Roleplay: API Setup Guide
• Uncensored LLM API: OpenAI-Compatible & JSON Mode