LangChain with OpenAI-Compatible API: Multi-Model Integration

Introduction

LangChain is the most popular framework for building LLM applications. Through AI API Hub's OpenAI-compatible endpoint, you get access to 50+ models without changing your LangChain code — just switch the model name.

Setup

pip install langchain langchain-openai openai

Basic LangChain Integration

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="gpt-4o",
    openai_api_key="your-api-key",
    openai_api_base="https://api.apiyihe.org/v1",
    temperature=0.7,
    max_tokens=1000,
)

response = llm.invoke("Explain LangChain in one paragraph")
print(response.content)

Switching Models Mid-Session

# Use GPT-4o for complex reasoning
llm.model = "gpt-4o"
complex_answer = llm.invoke("Analyze this financial report...")

# Switch to DeepSeek V3 for cost-effective bulk processing
llm.model = "deepseek-v3"
bulk_answers = llm.batch([f"Summarize: {doc}" for doc in documents])

# Use Claude for nuanced writing
llm.model = "claude-3.5-sonnet"
essay = llm.invoke("Write a thoughtful essay on AI ethics")

Prompt Templates

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a {role}. Respond in {language}."),
    ("user", "{input}")
])

chain = prompt | llm

response = chain.invoke({
    "role": "financial analyst",
    "language": "English",
    "input": "Explain the impact of AI on stock market predictions"
})

Chains with Multiple Models

from langchain_openai import ChatOpenAI

# Two LLM instances for different tasks
planner = ChatOpenAI(
    model="gpt-4o",
    openai_api_key="your-api-key",
    openai_api_base="https://api.apiyihe.org/v1",
)

executor = ChatOpenAI(
    model="deepseek-v3",
    openai_api_key="your-api-key",
    openai_api_base="https://api.apiyihe.org/v1",
)

# Plan with GPT-4o, execute with DeepSeek
plan = planner.invoke("Create a step-by-step plan to build a REST API")
code = executor.invoke(f"Write Python code for: {plan.content}")

Retrieval-Augmented Generation (RAG)

from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_community.vectorstores import Chroma
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser

# Embeddings through the same API
embeddings = OpenAIEmbeddings(
    model="text-embedding-3-small",
    openai_api_key="your-api-key",
    openai_api_base="https://api.apiyihe.org/v1",
)

# Vector store
vectorstore = Chroma.from_documents(documents, embeddings)
retriever = vectorstore.as_retriever()

# RAG chain
llm = ChatOpenAI(
    model="gpt-4o",
    openai_api_key="your-api-key",
    openai_api_base="https://api.apiyihe.org/v1",
)

def format_docs(docs):
    return "\n\n".join(doc.page_content for doc in docs)

rag_chain = (
    {"context": retriever | format_docs, "question": RunnablePassthrough()}
    | prompt
    | llm
    | StrOutputParser()
)

answer = rag_chain.invoke("What are the key findings?")

Agents with Tool Use

from langchain_openai import ChatOpenAI
from langchain.agents import AgentExecutor, create_openai_functions_agent
from langchain.tools import tool

@tool
def get_weather(location: str) -> str:
    """Get weather for a location"""
    return f"Weather in {location}: 22°C, sunny"

@tool
def calculate(expression: str) -> str:
    """Calculate a math expression"""
    return str(eval(expression))

llm = ChatOpenAI(
    model="gpt-4o",
    openai_api_key="your-api-key",
    openai_api_base="https://api.apiyihe.org/v1",
)

tools = [get_weather, calculate]
agent = create_openai_functions_agent(llm, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools)

result = agent_executor.invoke({
    "input": "What's the weather in Tokyo and what's 123 * 456?"
})

Common Errors

| Error | Fix | |---|---| | "Model not found" | Check model name spelling; use exact model ID | | Auth errors | Verify API key and base_url format | | Timeout | Increase request_timeout in ChatOpenAI | | Rate limit | Add rate limiter or switch to cheaper model for bulk |

FAQ

Q: Can I use LangChain with all 50+ models? A: Yes. Any model accessible through the OpenAI-compatible endpoint works with LangChain's ChatOpenAI.

Q: Does LangChain streaming work? A: Yes. Use llm.stream() or llm.astream() for streaming with all models.

Q: Can I use LangSmith for tracing? A: Yes. LangSmith traces work identically regardless of the model provider.