LLM Function Calling: Connect AI to Your APIs the Right Way

Why Function Calling Matters

An LLM on its own can only generate text. It can’t check your database, call your API, send an email, or look up today’s weather. Function calling (also called tool use) bridges this gap — it lets the model request specific actions that your code executes.

The pattern is universal across providers: you describe available functions as JSON schemas, the model decides when to call them and with what arguments, your code executes the function, and you send the result back for the model to synthesize into a final answer.

This is the foundation of every AI agent, chatbot with real-time data access, and AI-powered workflow automation. If you’re building anything beyond a basic text generator, you need this.

The Universal Pattern

Regardless of whether you use OpenAI, Anthropic, or open-source models, the flow is identical:

1. Define tools — describe your functions with names, descriptions, and parameter schemas
2. Send the conversation — include the tool definitions with your API call
3. Receive a tool call — the model returns a structured request instead of text
4. Execute the function — your code runs the requested function with the model’s arguments
5. Return results — send the function output back to the model
6. Get the final response — the model synthesizes the answer using the function results

Implementation With OpenAI

Here’s a complete, working example using the OpenAI Chat Completions API:

import OpenAI from 'openai';

const openai = new OpenAI();

// Step 1: Define your tools
const tools = [
  {
    type: 'function',
    function: {
      name: 'get_order_status',
      description: 'Retrieves the current shipping status and tracking number for a customer order. Use when the user asks about their order.',
      parameters: {
        type: 'object',
        properties: {
          order_id: {
            type: 'string',
            description: 'The order ID in format ORD-XXXX'
          }
        },
        required: ['order_id']
      }
    }
  },
  {
    type: 'function',
    function: {
      name: 'search_products',
      description: 'Searches the product catalog by keyword. Returns matching products with prices and availability.',
      parameters: {
        type: 'object',
        properties: {
          query: { type: 'string', description: 'Search keywords' },
          max_results: { type: 'integer', description: 'Maximum results to return (default 5)' }
        },
        required: ['query']
      }
    }
  }
];

// Step 2: Your actual function implementations
const functionMap = {
  get_order_status: async ({ order_id }) => {
    // In production, this queries your database
    return { order_id, status: 'shipped', tracking: 'TRK-98765', eta: '2026-07-18' };
  },
  search_products: async ({ query, max_results = 5 }) => {
    // In production, this queries your product API
    return [{ name: 'Wireless Keyboard', price: 79.99, in_stock: true }];
  }
};

// Step 3: The conversation loop
async function chat(userMessage) {
  const messages = [{ role: 'user', content: userMessage }];

  const response = await openai.chat.completions.create({
    model: 'gpt-4o',
    messages,
    tools
  });

  const choice = response.choices[0];

  // Check if the model wants to call a function
  if (choice.finish_reason === 'tool_calls') {
    // Execute each tool call
    for (const toolCall of choice.message.tool_calls) {
      const fn = functionMap[toolCall.function.name];
      const args = JSON.parse(toolCall.function.arguments);
      const result = await fn(args);

      // Add the tool call and result to the conversation
      messages.push(choice.message);
      messages.push({
        role: 'tool',
        tool_call_id: toolCall.id,
        content: JSON.stringify(result)
      });
    }

    // Get the final response with tool results
    const finalResponse = await openai.chat.completions.create({
      model: 'gpt-4o',
      messages,
      tools
    });

    return finalResponse.choices[0].message.content;
  }

  return choice.message.content;
}

Implementation With Anthropic (Claude)

The pattern is the same; the API shape differs slightly:

import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic();

const tools = [
  {
    name: 'get_weather',
    description: 'Gets the current weather for a city. Returns temperature, conditions, and humidity.',
    input_schema: {
      type: 'object',
      properties: {
        city: { type: 'string', description: 'City name (e.g., "London")' },
        units: { type: 'string', enum: ['celsius', 'fahrenheit'], description: 'Temperature unit' }
      },
      required: ['city']
    }
  }
];

async function chat(userMessage) {
  let messages = [{ role: 'user', content: userMessage }];

  const response = await client.messages.create({
    model: 'claude-sonnet-4-20250514',
    max_tokens: 1024,
    tools,
    messages
  });

  // Check for tool_use content blocks
  if (response.stop_reason === 'tool_use') {
    const toolUse = response.content.find(b => b.type === 'tool_use');
    const result = await executeFunction(toolUse.name, toolUse.input);

    // Send the result back
    messages.push({ role: 'assistant', content: response.content });
    messages.push({
      role: 'user',
      content: [{
        type: 'tool_result',
        tool_use_id: toolUse.id,
        content: JSON.stringify(result)
      }]
    });

    const finalResponse = await client.messages.create({
      model: 'claude-sonnet-4-20250514',
      max_tokens: 1024,
      tools,
      messages
    });

    return finalResponse.content[0].text;
  }

  return response.content[0].text;
}

Key differences: Anthropic uses input_schema instead of parameters, tool results are sent as tool_result content blocks within a user message, and tool invocations appear as tool_use content blocks in the assistant message.

Writing Good Tool Descriptions

The quality of your tool descriptions directly determines whether the model calls the right tool with the right arguments. Vague descriptions cause misrouting.

Include: what the function does, what it returns, format requirements for parameters, and any limitations.

Handling Multiple Tool Calls

Models can request multiple function calls in a single turn. OpenAI’s API supports parallel tool calls natively — the tool_calls array may contain several entries.

// User: "What's my order status for ORD-1234 and ORD-5678?"
// The model might return TWO tool calls at once

for (const toolCall of choice.message.tool_calls) {
  const fn = functionMap[toolCall.function.name];
  const args = JSON.parse(toolCall.function.arguments);
  
  // Execute in parallel for speed
  results.push(
    fn(args).then(result => ({ id: toolCall.id, result }))
  );
}

const resolved = await Promise.all(results);

Execute parallel tool calls concurrently with Promise.all(). Don’t run them sequentially — the user is already waiting for the model to respond.

Building an Agent Loop

For complex tasks, the model might need multiple rounds of tool calls. An agent loop handles this:

async function agentLoop(userMessage, maxIterations = 10) {
  const messages = [{ role: 'user', content: userMessage }];
  let iterations = 0;

  while (iterations < maxIterations) {
    const response = await openai.chat.completions.create({
      model: 'gpt-4o', messages, tools
    });

    const choice = response.choices[0];

    if (choice.finish_reason !== 'tool_calls') {
      return choice.message.content; // Done — return final text
    }

    messages.push(choice.message);

    for (const tc of choice.message.tool_calls) {
      const result = await executeFunction(tc.function.name, JSON.parse(tc.function.arguments));
      messages.push({ role: 'tool', tool_call_id: tc.id, content: JSON.stringify(result) });
    }

    iterations++;
  }

  return 'Agent reached maximum iterations.';
}

The maxIterations guard is critical. Without it, a confused model can loop forever, burning API credits. Always set a ceiling.

Common Mistakes

1. Not Validating Tool Arguments

The model generates arguments, not your code. Treat them like user input — validate everything before execution. An LLM might pass an unexpected type, a malformed ID, or a value outside your expected range.

2. Exposing Dangerous Functions

If you give the model a delete_user function, it will eventually call it. Only expose functions that are safe for the model to invoke. Add confirmation steps for destructive actions.

3. Too Many Tools at Once

Accuracy degrades when you present more than 20-30 tools. The model struggles to pick the right one. Use tool routing — dynamically select the 5-10 most relevant tools based on the user’s query before sending them to the model.

4. Ignoring Token Costs

Tool definitions consume input tokens on every API call. A large tool list can add thousands of tokens per request. Use prompt caching (available in both OpenAI and Anthropic) to reduce this cost.

Best Practices

• Write descriptions like documentation. Include parameter formats, return value descriptions, and edge cases.
• Always set a max iteration limit in agent loops.
• Validate all arguments before execution. Use schema validation libraries like Zod or Joi.
• Log every tool call. You need an audit trail of what the model requested and what your code executed.
• Use structured outputs (strict mode) when available to guarantee the model’s arguments match your schema exactly.
• Cache tool definitions using prompt caching to reduce token costs on repeated calls.

Wrapping Up

Function calling is the bridge between language models and real-world systems. The pattern is simple — define tools, let the model decide when to use them, execute the results, and feed them back. The implementation details vary between providers, but the architecture is universal.

Start with 2-3 well-described tools, build a robust execution layer with validation and logging, and add complexity only when you’ve proven the basic pattern works for your use case.

FAQ

Can the model call functions I haven’t defined?

No. The model can only request tools you’ve explicitly provided in the tools parameter. It cannot access arbitrary code, APIs, or system resources.

What if the model hallucinates a function call?

It can generate arguments that look plausible but are wrong — a non-existent order ID, for example. Your execution layer should handle this gracefully and return an error message the model can use to course-correct.

Is function calling the same as MCP?

No. Function calling is an API-level feature for connecting models to your code. The Model Context Protocol (MCP) is a standardized protocol for connecting models to external data sources and services. MCP can use function calling internally, but it adds discoverability, authentication, and transport layers on top.

How do I test function calling without making real API calls?

Mock your function implementations during development. Return static data from your tool functions. This lets you iterate on the prompt and tool descriptions without hitting rate limits or incurring costs.