Smoltalk

Smoltalk exposes a common API to different LLM providers. There are other packages that do this, but Smoltalk allows you to build strategies on top of it. Here is a simple example.

Install

pnpm install smoltalk
Copy

Hello world example

import { text, userMessage } from "smoltalk";

async function main() {
  const messages = [userMessage("Write me a 10 word story.")];
  const response = await text({
    messages,
    model: "gpt-5.4",
  });
  console.log(response);
}

main();
Copy

This is functionality that other packages allow.

{
  success: true,
  value: {
    output: 'Clock stopped; everyone smiled as tomorrow finally arrived before yesterday.',
    toolCalls: [],
    usage: {
      inputTokens: 14,
      outputTokens: 15,
      cachedInputTokens: 0,
      totalTokens: 29
    },
    cost: {
      inputCost: 0.000035,
      outputCost: 0.000225,
      cachedInputCost: undefined,
      totalCost: 0.00026,
      currency: 'USD'
    },
    model: 'gpt-5.4'
  }
}
Copy

What if you wanted to have fallbacks in case the OpenAI API was down? Just change the model field:

  const response = await text({
    messages,
    model: fallback("gpt-5.4", "gemini-2.5-flash-lite"),
    // or multiple fallbacks:
    // model: fallback("gpt-5.4", ["gemini-2.5-flash-lite", "gemini-3-flash-preview"]),
  });
Copy

Or what if you wanted to try a couple of models and take the first response?

  const response = await text({
    messages,
    model: race("gpt-5.4", "gemini-2.5-flash-lite", "o4-mini"),
  });
Copy

Or combine them:

  const response = await text({
    messages,
    model: race(fallback("gpt-5.4", "gemini-2.5-flash-lite"), "o4-mini"),
  });
Copy

You get the idea.

Longer tutorial

To use Smoltak, you first create a client:

import { getClient } from "smoltalk";

const client = getClient({
  openAiApiKey: process.env.OPENAI_API_KEY || "",
  googleApiKey: process.env.GEMINI_API_KEY || "",
  logLevel: "debug",
  model: "gemini-2.0-flash-lite",
});
Copy

Then you can call different methods on the client. The simplest is prompt:

const resp = await client.prompt("Hello, how are you?");
Copy

If you want tool calling, structured output, etc., text may be a cleaner option:

let messages: Message[] = [];
  messages.push(
    userMessage(
      "Please use the add function to add the following numbers: 3 and 5"
    )
  );
  const resp = await client.text({
    messages,
  });
Copy

Here is an example with tool calling:

function add({ a, b }: { a: number; b: number }): number {
  return a + b;
}

const addTool = {
  name: "add",
  description: "Adds two numbers together and returns the result.",
  schema: z.object({
    a: z.number().describe("The first number to add"),
    b: z.number().describe("The second number to add"),
  }),
};

const resp = await client.text({
  messages,
  tools: [addTool]
});

Copy

Here is an example with structured output:

const resp = await client.text({
  messages,
  responseFormat: z.object({
    result: z.number(),
  });
});
Copy

A couple of design decisions to note:

• You specify different API keys using different parameter names. This means you could set a couple of different API keys and then be able to change the model name without worrying about the keys, which makes things easier for code generation.
• The schema for tools and structured outputs is defined using Zod.
• Parameter names are camel case, as that is the naming convention in TypeScript. They are converted to snake case for you if required by the APIs.

Configuration Options

SmolPromptConfig is the union of client config (SmolConfig) and per-request config (PromptConfig). You can pass all options together to text(), or split them between getClient() and individual calls.

Client options (SmolConfig)

Request options (PromptConfig)

responseFormatOptions

Used with responseFormat to control validation behavior (currently OpenAI only).

toolLoopDetection

Detects when the model is stuck in a repetitive tool-call loop.

Limitations

Smoltalk has support for a limited number of providers right now, and is mostly focused on the stateless APIs for text completion, though I plan to add support for more providers as well as image and speech models later. Smoltalk is also a personal project, and there are alternatives backed by companies:

• Langchain
• OpenRouter
• Vercel AI

Contributing

Contributions are welcome. Any of the following contributions would be helpful:

• Adding support for API parameters or endpoints
• Adding support for different providers
• Updating the list of models