Use the vitals package with ellmer to evaluate and compare the accuracy of LLMs, including writing evals to test local models.

Roots looks for configuration at /etc/roots/config.yaml by default. You can specify an alternative path with the --config flag. This will prompt you for the essential settings and create the config ...

Install Python (above 3.5 is Recommended) Upgrade Pip and install setuptools, wheel and py2exe python -m pip install --upgrade pip python -m pip install -U pip ...