GitHub

NVIDIA/TensorRT-Edge-LLM

TensorRT Edge-LLM is NVIDIA's high-performance C++ inference runtime for Large Language Models (LLMs) and Vision-Language Models (VLMs) on embedded platforms. It enables efficient deployment of ...
GitHub

Self-hosted LLM evaluation studio with blind A/B/C judging and stakeholder-ready exports.

Avoid a $20K–$150K wrong model selection decision. BeLLMark produces the defensible evaluation artifact your procurement, legal, and technical teams need — with full statistical rigor, on your own ...
IEEE

First Experimental Demonstration of Disturb-Free 3D Vertical 1T-nC-1T Ferroelectric-based KV Cache with Co-Optimization of Hybrid Analog-Digital CIM and Token-Wise Dynamic ...

Abstract: For the first time, a ferroelectric (FE)-based key-value (KV) cache for large language models (LLMs) is proposed and experimentally demonstrated. Through device-architecture-algorithm ...
IEEE

Optimization and Benchmarking of Monolithically Stackable Gain Cell Memory for Last-Level Cache

Abstract: The Last Level Cache (LLC) is the processor’s critical bridge between on-chip and off-chip memory levels - optimized for high density, high bandwidth, and low operation energy. To date, high ...