“Give the Model a Layup”: How to Cut AI Cost Without Cutting Quality | Arjun Singh Jamwal (Verizon)

When AI does the heavy lifting inside your product, it changes the historic economics of software: you pay for that work every time a user touches it. For a funded team, that’s a line item. For a bootstrapped project, it can destroy the business.

Arjun Singh Jamwal is a Senior PM at Verizon, where he’s worked on Verizon Cloud and now Verizon Home. On the side, he’s bootstrapping Goodmuncher, an app that helps people eat fewer ultra-processed foods, so every token comes out of his own pocket.

His solution is to give the model a layup, as he puts it. Instead of having AI analyze every ingredient from scratch each time a user scans a label, Goodmuncher scores ingredients and products ahead of time and stores embeddings in a vector database. When a user looks something up, the model finds the answer that was already processed instead of analyzing from scratch each time. Same result for the user, a fraction of the inference cost for the company.

In this episode, Arjun gives a live demo of that approach with a pipeline that turns a single command into a 10-slide social carousel in about 30 seconds. Along the way, he shares:

• How better upfront prep lets smaller, cheaper options perform like frontier models and makes swapping models easier down the line
• Why he hard-codes the rules he can’t break instead of trusting the model to remember them
• His step-by-step framework for evaluating agents, from golden datasets to LLM-as-judge and observability tracing

Links

LinkedIn: https://www.linkedin.com/in/arjunjamwal/
Goodmuncher: https://goodmuncher.com/

Chapters

00:00 Introduction
02:16 Treating tokens as unpredictable OpEx vs. a One-time CapEx
06:21 Giving the model a layup by pre-processing data upfront
10:58 Arjun's demo: Turning one command into a 10-slide social carousel
15:44 Inside the pipeline architecture and its agents
19:09 How breaking up workflows cuts costs 30x and lowers user pricing
23:15 Arjun's framework for evaluating AI agents
27:56 Conclusion

Follow LaunchPod on YouTube

We have a new YouTube page! Watch full episodes of our interviews with PM leaders and subscribe!

What does LogRocket do?

LogRocket's Galileo AI watches user sessions for you and surfaces the technical and usability issues holding back your web and mobile apps. Understand where your users are struggling by trying it for free at LogRocket.com.