Expert evaluations are used when a domain expert is evaluating the output of your LLM. So, if your model is made to answer medicine-related questions, you can hire a group of doctors or medical experts to evaluate it. This provides the most reliable and meaningful foundation but has the downside of being very expensive.
In crowd-sourced (non-expert) eval…
Keep reading with a 7-day free trial
Subscribe to Text Generation to keep reading this post and get 7 days of free access to the full post archives.