Entropy

Entropy measures how uncertain something is before you know the outcome. A coin you are sure will land heads has zero entropy; a fair coin has one bit. The more possible answers there are, and the more evenly likely they are, the higher the entropy, and the more information it takes to resolve.

It underlies how we think about automation cost. A task with little uncertainty can be handled by fixed code, while an uncertain one needs intelligence and context to narrow it down. Retrieval and good prompts lower a task's entropy by supplying the missing information.