Tool Calling

Enable LLMs to interact with external systems using function/tool calling.

Tool calling (also known as function calling) allows LLMs to interact with external systems by generating structured output that your application can execute. This enables powerful integrations like database queries, API calls, calculations, and more.

Model capabilities

Not all models within a provider support tool calling. Check the model's documentation for specific capabilities. Generally, larger and newer models have better tool calling support.

Why tool calling matters

External integrations — Connect LLMs to databases, APIs, and services
Structured output — Get reliable JSON responses instead of free-form text
Multi-step reasoning — Let models break down complex tasks into tool calls
Real-time data — Access live information beyond the model's training cutoff

How it works

You define tools with a name, description, and JSON schema for parameters. The model decides when to call a tool based on the user's message, returning a tool_calls object with the function name and arguments.

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://api.aivene.com/v1',
  apiKey: process.env.AIVENE_API_KEY
});

const tools = [
  {
    type: 'function',
    function: {
      name: 'get_weather',
      description: 'Get current weather for a location',
      parameters: {
        type: 'object',
        properties: {
          location: {
            type: 'string',
            description: 'City name, e.g. "San Francisco"'
          },
          unit: {
            type: 'string',
            enum: ['celsius', 'fahrenheit'],
            description: 'Temperature unit'
          }
        },
        required: ['location']
      }
    }
  }
];

const response = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    { role: 'user', content: 'What is the weather in Tokyo?' }
  ],
  tools
});

console.log(response.choices[0].message.tool_calls);
// [{ id: 'call_abc123', type: 'function', function: { name: 'get_weather', arguments: '{"location":"Tokyo"}' } }]

Handling tool calls

When the model returns tool calls, you execute the functions and send the results back in a follow-up message.

const message = response.choices[0].message;

if (message.tool_calls) {
  const toolResults = await Promise.all(
    message.tool_calls.map(async (toolCall) => {
      const args = JSON.parse(toolCall.function.arguments);

      // Execute the actual function
      let result;
      if (toolCall.function.name === 'get_weather') {
        result = await fetchWeather(args.location, args.unit);
      }

      return {
        role: 'tool',
        tool_call_id: toolCall.id,
        content: JSON.stringify(result)
      };
    })
  );

  // Send tool results back to the model
  const finalResponse = await client.chat.completions.create({
    model: 'gpt-4o',
    messages: [
      { role: 'user', content: 'What is the weather in Tokyo?' },
      message,
      ...toolResults
    ],
    tools
  });

  console.log(finalResponse.choices[0].message.content);
  // "The weather in Tokyo is currently 22°C with partly cloudy skies."
}

Parallel tool calls

Models can request multiple tool calls in a single response. This is useful for queries that require data from multiple sources.

const response = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [
    { role: 'user', content: 'Compare weather in Tokyo and New York' }
  ],
  tools
});

// message.tool_calls may contain multiple calls:
// [
//   { function: { name: 'get_weather', arguments: '{"location":"Tokyo"}' } },
//   { function: { name: 'get_weather', arguments: '{"location":"New York"}' } }
// ]

Parallel execution

When you receive multiple tool calls, execute them in parallel using Promise.all() for better performance. Return all results in the same order as the tool calls.

Tool choice

Control how the model uses tools with the tool_choice parameter:

Value	Behavior
`"auto"`	Model decides whether to call a tool (default)
`"none"`	Model won't call any tools
`"required"`	Model must call at least one tool
`{ type: "function", function: { name: "..." } }`	Force a specific tool

// Force the model to use a specific tool
const response = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'Tell me about Paris' }],
  tools,
  tool_choice: { type: 'function', function: { name: 'get_weather' } }
});

Streaming with tools

Tool calls work with streaming responses. The tool call data arrives in chunks that you need to accumulate.

const stream = await client.chat.completions.create({
  model: 'gpt-4o',
  messages: [{ role: 'user', content: 'What is the weather in London?' }],
  tools,
  stream: true
});

let toolCalls: Record<number, { id: string; name: string; arguments: string }> = {};

for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta;

  if (delta?.tool_calls) {
    for (const tc of delta.tool_calls) {
      if (!toolCalls[tc.index]) {
        toolCalls[tc.index] = { id: tc.id ?? '', name: '', arguments: '' };
      }
      if (tc.id) toolCalls[tc.index].id = tc.id;
      if (tc.function?.name) toolCalls[tc.index].name = tc.function.name;
      if (tc.function?.arguments) toolCalls[tc.index].arguments += tc.function.arguments;
    }
  }
}

// Now execute tools with accumulated data
console.log(Object.values(toolCalls));

Best practices

Write clear descriptions — Tool and parameter descriptions help the model understand when and how to use each tool. Be specific about what the tool does and what format arguments should be in.
Use strict schemas — Define required fields and enums where possible to get more reliable outputs from the model.
Validate arguments — Always validate and sanitize tool arguments before executing them. The model may produce malformed or unexpected values.
Handle errors gracefully — If a tool fails, return an error message to the model so it can adjust its approach or inform the user.
Limit tool count — While you can define many tools, too many options can confuse the model. Group related functionality and keep the list focused.

Example: Multi-tool agent

Here's a complete example with multiple tools working together:

const tools = [
  {
    type: 'function',
    function: {
      name: 'search_products',
      description: 'Search for products in the catalog',
      parameters: {
        type: 'object',
        properties: {
          query: { type: 'string', description: 'Search query' },
          category: { type: 'string', description: 'Product category' },
          max_price: { type: 'number', description: 'Maximum price filter' }
        },
        required: ['query']
      }
    }
  },
  {
    type: 'function',
    function: {
      name: 'get_product_details',
      description: 'Get detailed information about a specific product',
      parameters: {
        type: 'object',
        properties: {
          product_id: { type: 'string', description: 'The product ID' }
        },
        required: ['product_id']
      }
    }
  },
  {
    type: 'function',
    function: {
      name: 'add_to_cart',
      description: 'Add a product to the shopping cart',
      parameters: {
        type: 'object',
        properties: {
          product_id: { type: 'string', description: 'The product ID' },
          quantity: { type: 'number', description: 'Quantity to add', default: 1 }
        },
        required: ['product_id']
      }
    }
  }
];

async function chat(userMessage: string, history: Message[]) {
  const messages = [...history, { role: 'user', content: userMessage }];

  while (true) {
    const response = await client.chat.completions.create({
      model: 'gpt-4o',
      messages,
      tools
    });

    const message = response.choices[0].message;
    messages.push(message);

    if (!message.tool_calls) {
      return message.content;
    }

    // Execute all tool calls
    const toolResults = await Promise.all(
      message.tool_calls.map(async (tc) => {
        const args = JSON.parse(tc.function.arguments);
        const result = await executeTool(tc.function.name, args);

        return {
          role: 'tool',
          tool_call_id: tc.id,
          content: JSON.stringify(result)
        };
      })
    );

    messages.push(...toolResults);
  }
}

Common patterns

Retrieval-augmented generation (RAG)

Use tools to fetch relevant documents before generating a response:

const tools = [{
  type: 'function',
  function: {
    name: 'search_knowledge_base',
    description: 'Search internal documentation and knowledge base',
    parameters: {
      type: 'object',
      properties: {
        query: { type: 'string', description: 'Search query' }
      },
      required: ['query']
    }
  }
}];

Code execution

Let the model run code in a sandboxed environment:

const tools = [{
  type: 'function',
  function: {
    name: 'execute_python',
    description: 'Execute Python code and return the output',
    parameters: {
      type: 'object',
      properties: {
        code: { type: 'string', description: 'Python code to execute' }
      },
      required: ['code']
    }
  }
}];

Database queries

Enable natural language to SQL conversion:

const tools = [{
  type: 'function',
  function: {
    name: 'query_database',
    description: 'Execute a read-only SQL query on the database',
    parameters: {
      type: 'object',
      properties: {
        sql: { type: 'string', description: 'SQL SELECT query' }
      },
      required: ['sql']
    }
  }
}];

Limitations

No direct execution — The model generates tool calls but cannot execute them directly. Your application must handle execution.
Token overhead — Tool definitions count toward input tokens. Large schemas increase costs.
Hallucinated arguments — Models may invent plausible-looking but invalid argument values. Always validate inputs.
Context window — Long tool results consume context space. Consider summarizing or truncating large responses.