Building an iPhone App for my toddler with Claude Code: programming that used to give me headaches is now so simple

I initially thought my idea of building an iPhone app was crazy, but I’m glad I tried. Getting it done in such a short time made me really happy.

Many years ago, I took a several-week iOS programming course where we built simple apps each week. I remember struggling with just learning syntax, following along with demos, and calling APIs - it made my head spin after creating just a few simple pages. Each assignment took me forever to barely complete, I felt like my brain was going to explode. Honestly, all those details about page navigation and Xcode environment setup -- I would forget everything right after learning it. That’s why I’m grateful for myself to switch careers — though I studied computer science, I never became a programmer after graduation. While I’m fascinated by cutting-edge technology and can read code, actually programming and debugging from scratch was truly painful for me.

I found some old screenshots of apps I made during that iOS course. Looking at them brought back those painful programming memories:

I wasn’t expecting that after all these years, I would be able to build a small but complete iPhone app using Claude Code AI in just three nights — about 5-6 hours total. This app was made for my two-year-old to play with. She uses the camera to identify objects, and the app reads out what it recognizes. The app uses a small model for object recognition that’s quite limited with pretty poor accuracy, but she’s having a blast with it! She points her phone at our dog and fruits, carrying it around the house everywhere.

Watching my toddler play with this app was really interesting. I wasn’t sure if she would get bored or frustrated with the inaccurate object recognition — the same way I felt while building it. But to my surprise, she thought the mistakes were hilarious. For instance, when the app identified our little dog as a cat, instead of being disappointed, she burst into laughter and found it amusing. This made me realize that adults might think “Ugh, this recognition is so inaccurate, this app is useless,” so I totally didn’t expect her reaction to be completely different from mine. I guess that’s the value of user testing - sometimes users and product makers think very differently.

Building AI projects: taking small steps

Previously, I had experimented with ChatGPT, Cursor, and Visual Studio with AI plugins to build small projects like a simple web game, a Chinese character tracing React app, and a tic-tac-toe game on hardware. I experienced the amazing power of AI-assisted programming. This past year, AI programming tools have developed rapidly with comprehensive improvements in both technology and user experience. Recently, my husband has been enthusiastically using Anthropic’s Claude Code, so I wanted to try it too. This was my first experience with an AI programming tool that works entirely in the command line. It can fully control my project folder with incredibly high efficiency — I was blown away all over again.

I recently read advice from Andrew Ng, an AI pioneer, about building small AI projects, and it really clicked with me. He said that using AI to help with programming is now very common, so many ideas are much easier to implement than before. Don’t try to build something huge right away - start small, the smaller the better. Get it built first - we’ll feel accomplished, and if we want to modify or expand it, AI programming makes changes quite fast.

My app follows exactly this approach — its final functionality is small and simple enough. Open the camera, the app invites little ones to point the camera at different objects, then it automatically identifies items and displays the recognition results with confidence scores. Since she can’t read yet, I added Chinese-English voice switching and front/back camera switching. Just this simple little thing.

Actually, I initially wanted to make it more complex. If the object recognition part worked well, I wanted it to have conversations with my toddler, so she could learn about objects through interacting with the app, which would be more fun. But during testing, I found the small model YOLOv8 was really too limited - it can recognize at most around 80 types of objects. Many things couldn’t be successfully detected — it knew even less than my two-year-old daughter. Obviously there was no point making it more complex, let alone adding conversation features. As for switching to a different underlying model for more functionality, that’s for future versions.

I’m so grateful AI has made this process much more accessible and efficient, allowing me to quickly bring ideas to life. Using AI to build and test iteratively, discovering problems, then thinking about improvements — I really enjoy this process. I initially thought my idea of building an iPhone app was crazy, but I’m glad I tried. Getting it done in such a short time made me really happy.

The development experience

Regarding my specific development experience, Claude Code feels refreshingly different compared to traditional IDE programming environments. Although my interaction with Claude Code happens entirely in the command line without any other interface, I’m amazed by how deeply it understands my project files and how efficient it is. I allowed it to automatically code and save files, and also let it make any changes it deemed necessary in this project.

Below shows Claude Code listing its tasks and asking if I want to allow it to autonomously create folders. I chose the second option -- always allow:

Below shows Claude Code modifying existing code, marking the changes:

I didn’t look at the code throughout the process — I just told the AI my specific requirements, and it handled everything. If we think of this app-building process in terms of role-playing, I acted as the product manager and designer, while Claude Code was my software engineer completing all programming tasks. However, it can’t manipulate, Xcode the iOS programming environment, so I needed to manually add the project in Xcode, do necessary configurations, and feed AI error messages to fix the code.

The image below shows me following Claude Code’s guidance in the command line (left) to modify configurations in Xcode (center). When I couldn’t find what needed to be changed, I opened a new conversation in the Claude.ai interface (right) to ask, and with its help I found the solution. Actually, I could have asked directly in Claude Code, but I didn’t want extra questions to interrupt the guidance text it was showing me, so I just used another AI interface.

I had some amazing experiences - I never knew I could drag images directly into a command line, but Claude Code supports this! I made screenshot problems and dropped them in, and it could read the text in images and solve the issues. For larger tasks, it automatically created task lists and executed them step by step like a very organized programmer. The image below shows Claude Code working through its tasks one by one. I took a break during this time and came back to check its work:

Below is Claude Code telling me the project is ready for iPhone testing:

The longest time I spent on this project was probably configuring Xcode correctly. Xcode has some very counter-intuitive design that still requires manual intervention. Just figuring out which folder directory to put the model in and finding various configuration locations was maddening, but fortunately I had my AI companion, and I eventually got answers to all these questions from AI.

What will the world my toddler faces look like?

Since having my daughter, I love observing her behavioral reactions in various life scenarios and her interactions with my husband and me. I often wonder if I can create interesting things for her to play with. This is one of my great joys in parenting.

I also think about what kind of world she will face in the future. What kind of environment will her generation grow up in? What tools will they have? What will they be able to create? Maybe in the future they’ll think building an app like this is a piece of cake, but today we’re still exploring together with Generative AI that’s only a few years old.

I used to hear people say “I’ve walked more bridges than you’ve walked roads.” Those experiences and knowledge might have their value to some extent and can serve as reference, but to me, that’s a way of viewing the world as unchanging. Every era brings many changes, and every generation has new opportunities and experiences. No one knows what the future world will look like.

This is why I’m so curious about the future and can maintain an open mindset for learn new things and self-growth. Maybe this mindset can help me maintain a closer relationship with my daughter as she grows up, staying closer to her heart. At the same time, I can better understand what the new future world will actually look like, rather than just clinging to “back in my day…” or always missing the old world.