Let's help help help devs.
Target: <1000ms. Several important LLM CLI tools take multiple seconds. PRs welcome ❤️
| tool | cold | warm | version |
|---|---|---|---|
tensorrt-llm --help | 30726ms | 10853ms | 1.2.1 |
vllm --help | 18643ms | 6870ms | 0.25.1 |
VLMEvalKit --help | 14993ms | 4480ms | v0.2 |
sglang --help | 14614ms | 4444ms | 0.5.9 |
transformers --help | 7657ms | 2166ms | 5.14.1 |
datasets --help | 2709ms | 694ms | 5.0.0 |
llm --help | 1451ms | 646ms | 0.31.1 |
openai --help | 959ms | 421ms | 2.34.0 |
hf --help | 844ms | 331ms | 1.23.0 |
lm-eval --help | 1396ms | 229ms | 0.4.12 |
langchain-cli --help | 684ms | 209ms | 0.0.37 |
tokenspeed --help | 477ms | 178ms | 0.1.0@f3d14a8 |
ollama --help | 12ms | 12ms | 0.32.0 |
llama.cpp --help | 15ms | 11ms | b10042 |