流式输出
所有 ChatModel 都实现了 Runnable 接口，它自带所有方法的默认实现，即 ainvoke、batch、abatch、stream、astream。这让所有 ChatModel 都拥有基础的流式输出支持。
流式输出默认返回一个包含单个值的 Iterator（异步流式时则为 AsyncIterator），即底层 ChatModel 提供方返回的最终结果。这显然不会给你逐令牌（token-by-token）的流式输出——后者需要 ChatModel 提供方的原生支持——但它能确保你期望迭代器返回令牌的代码可适用于我们任意一个 ChatModel 集成。
在 此处 查看哪些集成支持逐令牌流式输出。
from langchain_community.chat_models import ChatAnthropicchat = ChatAnthropic(model="claude-2")
for chunk in chat.stream("Write me a song about goldfish on the moon"):
print(chunk.content, end="", flush=True) Here's a song I just improvised about goldfish on the moon:
Floating in space, looking for a place
To call their home, all alone
Swimming through stars, these goldfish from Mars
Left their fishbowl behind, a new life to find
On the moon, where the craters loom
Searching for food, maybe some lunar food
Out of their depth, close to death
How they wish, for just one small fish
To join them up here, their future unclear
On the moon, where the Earth looms
Dreaming of home, filled with foam
Their bodies adapt, continuing to last
On the moon, where they learn to swoon
Over cheese that astronauts tease
As they stare back at Earth, the planet of birth
These goldfish out of water, swim on and on
Lunar pioneers, conquering their fears
On the moon, where they happily swoon