inference
21 repositories with 10k+ stars are tagged inference.
Open in the explorer to search, filter and sort all of them.
- 1
A high-throughput and memory-efficient inference and serving engine for LLMs
- 2
Ultralytics YOLOv5 in PyTorch for object detection, instance segmentation, classification, training, and export.
- 3
Port of OpenAI's Whisper model in C/C++
- 4
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
- 5
Making large AI models cheaper, faster and more accessible
- 6
SGLang is a high-performance serving framework for large language models and multimodal models.
- 7
Cross-platform, customizable ML solutions for live and streaming media.
- 8
Faster Whisper transcription with CTranslate2
- 9
ncnn is a high-performance neural network inference framework optimized for the mobile platform
- 10
Machine Learning Engineering Open Book
- 11
- 12
🎨 The exhaustive Pattern Matching library for TypeScript, with smart type inference.
- 13
FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently.
- 14
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
- 15
- 16
The Triton Inference Server provides an optimized cloud and edge inferencing solution.
- 17
OpenVINO™ is an open source toolkit for optimizing and deploying AI inference
- 18
Example 📓 Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using 🧠 Amazon SageMaker.
- 19
Large Language Model Text Generation Inference
- 20
PyTorch implementation of YOLOv3, YOLOv3-SPP, and YOLOv3-tiny for real-time object detection with training, validation, inference, and multi-format export.
- 21
Production ready toolkit to run AI locally