GitHub - pytorch/executorch: On-device AI across mobile, embedded and edge for PyTorch
On-device AI inference powered by PyTorch
ExecuTorch is PyTorch's unified solution for deploying AI models on-device—from smartphones to microcontrollers—built for privacy, performance, and portability. It powers Meta's on-device AI across Instagram, WhatsApp, Quest 3, Ray-Ban Meta Smart Glasses, and
.
Deploy LLMs, vision, speech, and multimodal models with the same PyTorch APIs you already know—accelerating research to production with seamless model export, optimization, and deployment. No manual C++ rewrites. No format conversions. No vendor lock-in.
📘 Table of Contents
Why ExecuTorch?
🔒 Native PyTorch Export — Direct export from PyTorch. No .onnx, .tflite, or intermediate format conversions. Preserve model semantics.
⚡ Production-Proven — Powers billions of users at
Meta with real-time on-device inference
.
💾 Tiny Runtime — 50KB base footprint. Runs on microcontrollers to high-end smartphones.
🚀
— Open-source acceleration for Apple, Samsung, Qualcomm, ARM, MediaTek, Vulkan, and more.
🎯 One Export, Multiple Backends — Switch hardware targets with a single line change. Deploy the same model everywhere.
How It Works
ExecuTorch uses ahead-of-time (AOT) compilation to prepare PyTorch models for edge deployment:
🧩 Export — Capture your PyTorch model graph with torch.export()
⚙️ Compile — Quantize, optimize, and partition to hardware backends → .pte
🚀 Execute — Load .pte on-device via lightweight C++ runtime
Models use a standardized
.
delegate subgraphs to specialized hardware (NPU/GPU) with CPU fallback.
Learn more:
•
Quick Start
Installation
pip install executorchBackend export tools can require optional dependencies. For example, use pip install 'executorch[ethos_u]' for Ethos-U AOT export. Embedded toolchains, simulators, and target runtimes are installed separately.
For platform-specific setup (Android, iOS, embedded systems), see the
documentation for additional info.
Export and Deploy in 3 Steps
importtorchfromexecutorch.exirimportto_edge_transform_and_lowerfromexecutorch.backends.xnnpack.partition.xnnpack_partitionerimportXnnpackPartitioner# 1. Export your PyTorch modelmodel=MyModel().eval() example_inputs= (torch.randn(1, 3, 224, 224),) exported_program=torch.export.export(model, example_inputs) # 2. Optimize for target hardware (switch backends with one line)program=to_edge_transform_and_lower( exported_program, partitioner=[XnnpackPartitioner()] # CPU | CoreMLPartitioner() for iOS | QnnPartitioner() for Qualcomm ).to_executorch() # 3. Save for deploymentwithopen("model.pte", "wb") asf: f.write(program.buffer) # Test locally via ExecuTorch runtime's pybind API (optional)fromexecutorch.runtimeimportRuntimeruntime=Runtime.get() method=runtime.load_program("model.pte").load_method("forward") outputs=method.execute([torch.randn(1, 3, 224, 224)])Run on Device
#include<executorch/extension/module/module.h> #include<executorch/extension/tensor/tensor.h> Module module("model.pte"); auto tensor = make_tensor_ptr({2, 2}, {1.0f, 2.0f, 3.0f, 4.0f}); auto outputs = module.forward(tensor);
import ExecuTorch letmodule=Module(filePath:"model.pte")letinput=Tensor<Float>([1.0,2.0,3.0,4.0], shape:[2,2])letoutputs=try module.forward(input)
val module =Module.load("model.pte") val inputTensor =Tensor.fromBlob(floatArrayOf(1.0f, 2.0f, 3.0f, 4.0f), longArrayOf(2, 2)) val outputs = module.forward(EValue.from(inputTensor))LLM Example: Llama
Export Llama models using the
script or
:
# Using export_llm python -m executorch.extension.llm.export.export_llm --model llama3_2 --output llama.pte # Using Optimum-ExecuTorch optimum-cli export executorch \ --model meta-llama/Llama-3.2-1B \ --task text-generation \ --recipe xnnpack \ --output_dir llama_modelRun on-device with the LLM runner API:
#include<executorch/extension/llm/runner/text_llm_runner.h>auto runner = create_llama_runner("llama.pte", "tiktoken.bin"); executorch::extension::llm::GenerationConfig config{ .seq_len = 128, .temperature = 0.8f}; runner->generate("Hello, how are you?", config);
import ExecuTorchLLM letrunner=TextRunner(modelPath:"llama.pte", tokenizerPath:"tiktoken.bin")try runner.generate("Hello, how are you?",Config{ $0.sequenceLength =128}){ token inprint(token, terminator:"")}Kotlin (Android) —
•
val llmModule =LlmModule("llama.pte", "tiktoken.bin", 0.8f) llmModule.load() llmModule.generate("Hello, how are you?", 128, object:LlmCallback { overridefunonResult(result:String) { print(result) } overridefunonStats(stats:String) { } })For multimodal models (vision, audio), use the
which extends the LLM runner to handle image and audio inputs alongside text. See
and
examples.
See
for complete workflow including quantization, mobile deployment, and advanced options.
Next Steps:
📖
— Complete walkthrough for your first model
⚡
— Try ExecuTorch instantly in your browser
🤖
— LLM workflow with quantization and mobile demos
Platform & Hardware Support
PlatformSupported BackendsAndroidXNNPACK, Vulkan, Qualcomm, MediaTek, Samsung ExynosiOSXNNPACK, CoreML (Neural Engine)Linux / WindowsXNNPACK, OpenVINO, CUDA (experimental)macOSXNNPACK, Metal (experimental), MLX (experimental)Embedded / MCUXNNPACK, ARM Ethos-U, NXP, Cadence DSPSee
for detailed hardware requirements and optimization guides. For desktop/laptop GPU inference with CUDA and Metal, see the
. For Zephyr RTOS integration, see the
.
Production Deployments
ExecuTorch powers on-device AI at scale across Meta's family of apps, VR/AR devices, and partner deployments.
Examples & Models
LLMs:
,
,
,
Multimodal:
(vision-language),
(audio-language),
(vision-language)
Vision/Speech:
,
,
,
,
Resources:
directory •
out-of-tree demos •
for HuggingFace models •
for fine-tuned LLM deployment
Key Features
ExecuTorch provides advanced capabilities for production deployment:
Quantization — Built-in support via
for 8-bit, 4-bit, and dynamic quantization
Memory Planning — Optimize memory usage with ahead-of-time allocation strategies
Developer Tools — ETDump profiler, ETRecord inspector, and model debugger
Selective Build — Strip unused operators to minimize binary size
Custom Operators — Extend with domain-specific kernels
Dynamic Shapes — Support variable input sizes with bounded ranges
See
for quantization techniques, custom backends, and compiler passes.
Documentation
— Complete guides and tutorials
— Python, C++, Java/Kotlin APIs
— Build custom hardware backends
— Common issues and solutions
Community & Contributing
We welcome contributions from the community!
💬
— Ask questions and share ideas
🎮
— Chat with the team and community
🐛
— Report bugs or request features
🤝
— Guidelines and codebase structure
Citing ExecuTorch
If you found ExecuTorch helpful in your research and would like to acknowledge it, please cite us using the following BibTeX:
@article{executorch2026, title={{ExecuTorch} - A Unified {PyTorch} Solution to Run {AI} Models On-Device}, author={Nachin, Mergen and Desai, Digant and Jia, Sicheng Stephen and Lai, Chen and Liu, Mengwei and Szwejbka, Jacob and Alvarez, Raziel and Ascani, RJ and Bort, Dave and Candales, Manuel and others}, journal={arXiv preprint arXiv:2605.08195}, url={https://github.com/pytorch/executorch}, year={2026} }License
ExecuTorch is BSD licensed, as found in the
file.
———