discover · compare · collections · requirements · contribute · feed · watching · stats · help with your tokens
BlinkDL/RWKV-LM
RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.
History and source changes · Reports and owner responses · Compare
Similar
- EleutherAI/gpt-neo
An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.
Why this match
topics gpt, gpt-2, gpt-3; license differs: Apache-2.0 → MIT
- BlinkDL/ChatRWKV
ChatRWKV is like ChatGPT but powered by RWKV (100% RNN) language model, and open source.
Why this match
topics chatgpt, language-model, pytorch
- OptimalScale/LMFlow
An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.
Why this match
topics chatgpt, deep-learning, language-model
- EleutherAI/gpt-neox
An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries
Why this match
topics gpt-3, language-model, transformers
- microsoft/LoRA
Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"
Why this match
topics deep-learning, gpt-2, gpt-3; license differs: Apache-2.0 → MIT
- NVIDIA/FasterTransformer
Transformer related optimization, including BERT, GPT
Why this match
topics gpt, pytorch, transformer; language differs: Python → C++
- rasbt/LLMs-from-scratch
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
Why this match
topics attention-mechanism, deep-learning, gpt; language differs: Python → Jupyter Notebook
- codertimo/BERT-pytorch
Google AI 2018 BERT pytorch implementation
Why this match
topics language-model, pytorch, transformer