server : add SWA checkpoints by ggerganov · Pull Request #15293 · ggml-org/llama.cpp · GitHub

github.com

Tin mới

gg/server-swa-checkpoints

GitHub CopilotWrite better code with AI

GitHub Copilot appDirect agents from issue to merge

ActionsAutomate any workflow

CodespacesInstant dev environments

IssuesPlan and track work

Code ReviewManage code changes

Code QualityEnforce quality at merge

GitHub Advanced SecurityFind and fix vulnerabilities

Code securitySecure your code as you build

Secret protectionStop leaks before they start

View all use cases

Financial services

View all industries

Software Development

GitHub SponsorsFund open source developers

Copilot for BusinessEnterprise-grade AI features

Premium SupportEnterprise-grade 24/7 support

Pull requests 1.6k

Security and quality 13

libllama API changes

The server now makes checkpoints of the SWA memory in order to minimize the amount of context reprocessing. A SWA checkpoint represents the state (both the KV cells and KV data) of the cache. Only the SWA part is stored

llama.cpp/tools/server/server.cpp

server : add SWA checkpoints

View reviewed changes

Learn more about hiding disruptive comments

The reason will be displayed to describe this comment to others. Learn more.

cont : server clean-up

llama.cpp/include/llama.h

server : handle state restore fails

llama : add extended llama_state_seq_ API

server : do not make checkpoints if --swa-full

View reviewed changes

llama : remove flags value for NONE

server : configure number of SWA checkpoints with CLI arg

View reviewed changes

args : fix scope of new argument

View reviewed changes

changelog : libllama API #9289

Add GPT-OSS from OpenAI - closed in favor of 689 ikawrakow/ik_llama.cpp#683

Eval bug: Nemotron v2 Nano always reprocesses prompt #15677

implement context checkpointing for hybrid and recurrent models #16382

changelog : libllama API COG-GTM/llama.cpp#246

server : add SWA checkpoints (#15293)

Qwen3.5 waiting 3 seconds after generating. JamePeng/llama-cpp-python#77

server: fix multi-turn cache reuse for hybrid/recurrent models #20428

server : add SWA checkpoints (

server : add SWA checkpoints (

server : add SWA checkpoints (

server: merge custom preset aliases into existing HF cached model entries #23952

server : add SWA checkpoints (

server : add SWA checkpoints (#15293)