Hey there,

You're used to me talking about agent building and vibe coding, but I recently wrapped a project where I build a production AI feature in a consumer travel app for thousands of users! I wrote a deep dive about it here on LinkedIn, but want to talk to you all about how the skills I gained in building agents transferred to building a new feature for a consumer app.

First, what is Pangea? An app to share your travel plans, recommendations, and stats with friends. I've been working with them as an advisor for a few months now. We wanted to incorporate AI thoughtfully and had a few ideas on how we might do so.

We started with a feature to help users prepare for trips. We'll send AI written emails that are personalized to the user - accounting for their travel preferences, social network, and answer questions that they wouldn't have known to ask (e.g. if Uber doesn't work and they should download a different rideshare app before they land, if there are currency exchange protocols unique to a country, etc.) And we want these emails to sound like they're coming from a friend.

With a wide range of users with varying levels of data in the app, how do we make sure the emails add value, don't send incorrect information, and are personalized properly?

• Build internal traveler profiles based on user data + inferences made from their travel history
• Run evals over and over. Run more evals than you think. We even built an eval tool that allowed anyone to test the outputs that the models would generate in the app
• Root cause any errors, rather than simply adding more rules to your prompts. For example, let's say the outputs have a phrase you don't like. If you have the model self-improve, it'll probably add a rule that says "don't use x phrase". Instead, ask the model to find out why it came up with that phrase from the original prompt and tell it to improve the underlying prompt.

We'll be building more into this feature, and I'll share about it here. If you want a deeper dive, check out this article I wrote. And download the Pangea app, add a trip, and you'll see it in action. I'd appreciate any feedback.

— Shaalin