HomeArchive
Knowledge Hub
Paper Archive
Papers the group has read, presented, and reproduced — independent of the nomination cycle. Each entry includes session notes, reproduction code, and a vibe score from the group.
2
Papers
2
Reproduced
2
Venues
82%
Avg Vibe
TransformersNLPFoundations
ReproducedNeurIPS 2017 · Duc Vo
Attention Is All You Need
- ›Self-attention replaces recurrence, enabling full parallelization during training
- ›Multi-head attention lets the model jointly attend to information from different representation subspaces
- ›Positional encodings inject sequence order without recurrent structure
- ›Achieved SOTA on WMT 2014 EN-DE translation with far less compute than previous models
⚡ 85% hackable
LLMRLReasoning
ReproducedarXiv 2025 · Duc Vo
DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
- ›Pure RL with a simple accuracy + format reward produces emergent reasoning without any SFT warm-up
- ›GRPO (Group Relative Policy Optimization) stabilizes training without a value network
- ›R1-Zero shows self-reflection and multi-step reasoning emerging spontaneously from RL
- ›Distilled 7B/8B models are competitive with o1-mini on MATH-500 and LiveCodeBench
⚡ 78% hackable