Napkin Math: Are chatbots for survey automatization worth it?
Published on October 19, 2024 llm automatization napkin-mathLet’s say you’re a company doing surveys, gathering data for politicians. Let’s do some napkin math to see if the switch to voice-based chatbots is worth it.
Formula we will use to estimate operating costs:
costs = rent + energy consumption + n * minimum_wage + m *
(minimum_wage + something)
There’s more to running a company, but the elements listed are important because they will be radicaly changed by the switch to automated chatbots:
- you won’t need a full office for your
n
minimum wagers working in the call center - without an office, energy consumption will also be null (but you will need to pay for GPUs)
Voice generation cost
Elevenlabs seems like the goto service nowadays. Here’s their pricing (I won’t be including the free and stater tier):
Plan | Monthly cost | Monthly Credits Included | TTS minutes | Extra credits cost |
---|---|---|---|---|
Creator | $22 | 100,000 | 100 | $0.3 / 1000 |
Pro | $99 | 500,000 | 500 | $0.24 / 1000 |
Scale | $330 | 2,000,000 | 2,000 | $0.18 / 1000 |
Business | $1320 | 11,000,000 | 11,000 | $0.1 / 1000 |
Cost of a conversation
The formula for estimating the cost of an conversation would look something like if we take into account that:
- OpenAI pricing 1M input tokens at $2.5, 1M output tokens at $10
- the average cost of 1M tokens being ($2.5 + $10) / 2
- twillios $0.01 per minute
- the 0.75 tokens per word rule of thumb
- the average conversational speaking rate at 130 wpm
$$ token_{cost} = (2.5 + 10) * 10^{-6} * \frac{1}{2} $$
$$ f(minutes) = minutes * (130wpm * 0.75 * token_{cost} + 0.01 + tts_{cost}) $$
The cost of a single token is so small it’s not even worth
taking into account when estimating the cost of a voice call.
Simplified formula:
$$ f(minutes) = ($0.01 + tts_{cost}) * minutes$$
And 1M tokens would give us about $\frac{10^6}{130wpm}$ 128 hours of conversation, or 384 surveys.
Human element
We can safely assume people will not want to converse with chatbots, so it would be beneficial to allow people to choose a human agent to talk to. So you should not fire all your minimum wagers.
Conclusion
Yes, automatization is profitable. But you will be hated by users if you use the napkin math above to automatize your customer support center. AI Chatbots can’t automatize everything.