Toys to Stories: Bilingual Learning Through Play Time Storytelling
Empower bilingual families in supporting their children’s early bilingual language development through turning toys into interactive storytelling.
Over 21% of U.S. households speak a language other than English, yet many bilingual parents struggle to maintain their child’s mother tongue. Ages 0–5 are critical for language development, making early exposure essential (Kuhl, 2010). Toys to Stories helps by turning everyday toys into interactive companions, making bilingual storytelling easy and playful for busy parents.
Disclosure: This platform is currently in active development. You may encounter occasional bugs or UI inconsistencies, and some responses may be affected by the API.
Bilingual parents struggle to preserve their child’s heritage language in an English-dominated country.
Target User
👤
Demographics
Bilingual parents with children aged 0–5 in the early stages of language development.
💪🏻
Motivations
Parents aim to nurture bilingualism by actively integrating their heritage language into daily life, hoping to pass down culture, identity, and connection to family roots.
Pain Points that Parents Struggle With:
😵💫
Hard to Maintain Mother Tongue
In an English-dominant environment, parents find it hard to maintain consistent use of their heritage language.
😮💨
Too Tired to Be Creative
After a long day, they often lack the time or energy to create engaging, spontaneous stories for their children.
Parents Care About Education, Children Care About Joy
In order to understand parent's and children's needs, I explored existing solutions, talked to bilingual families, and reviewed research on early language development to inform my design.
Methodology:
📖
Literature Review
Explored research on early bilingual learning and how storytelling supports vocabulary growth.
🥊
Prior Art & Competitor Analysis
Reviewed language learning tools, AI story generators, and interactive toys for features, engagement, and parent effort.
🎤
Semi-Structured Interview
Spoke with 1 expert and 4 bilingual parents to uncover pain points, goals, and routines in language learning.
Research Insights
Lack of Time, Engagement & Ease
⏰
Lack of Time
Parents are mentally and physically drained after work and household responsibilities, leaving little bandwidth to come up with a story.
"After finishing work, cooking dinner, and doing housework, I just don’t have the energy or mental bandwidth to come up with stories." -P1
🧸
Lack of Engagement
Children often found existing bilingual resources boring or disconnected from their interests, making it difficult to maintain their attention.
"We’re trying to raise our daughter bilingual, but she only speaks English...I even had to hide English version of Dragons Love Tacos so she’d choose the Spanish version just to encourage her to engage with Spanish." -P2
✍🏻
Lack of Ease
Some parents already use ChatGPT for making stories, but without prompting skills, results were inconsistent and frustrating to repeat.
"I already use ChatGPT to tell my son stories based on his favorite toys, but it’s such a redundant process...I have to start from scratch every time." -P3
Language learning is more effective when built on toys children already love.
Children are emotionally attached to their favorite toys. Turning these toys into bilingual stories helps them learn through comfort and play. Storytelling is effective for language learning because it is already part of their daily routine (Fortuna et al., 2014; Grolig et al., 2020).
Helping bilingual families support early language development by turning toys into interactive stories.
How might we help bilingual parents seamlessly integrate storytelling into daily play to strengthen their child's early language acquisition and foster consistent bilingual engagement?
Balancing Human & AI Creation
❌ Should not:
Let AI Do All the Work.
✅ Should:
Use AI as an Assistant,
Not the Author.
Disclosure: Not real depiction of Drake
Designing Prompts for Purposeful Learning & Play
Final Prompt Design
Improved Prompt Accuracy and Output
🎯
More Accurate: Each step stays focused, reducing errors.
✨
Cleaner Context: Keeps inputs clear and avoids mixing stages.
🐛
Easier to Debug: Fix or improve one section without affecting the rest.
Here's the Process
Goal of Prompt
Our First Attempt at Story Generation
🧸
Create imaginative, toy-based stories using uploaded images as inspiration for characters.
📚
Teach language and a life lesson that has a valuable moral or social skill to young children.
Usability Test
Prompt Usability Findings
Focused on:
Story quality
Emotional tone
Usability
Desirability
Findings:
Story Structure: Needed preview, adjustable length, and word customization
Guardrails: Block child-face uploads
IP Sensitivity: Avoid references to famous characters like Winnie the Pooh
Potentially Infringed Laws
Staying Legally Safe with AI-Generated Content
I discovered the prompt could potentially violate copyright or trademark laws. To better understand the risks, I interviewed an expert in IP law, who explained that using characters like Winnie the Pooh could create unauthorized derivative works and lead to brand confusion.
Prompt Design Improvements
Refining the Prompt for Kids, Parents, and AI
Based on the feedbacks, I improved the prompt to generate better-structured stories, designed to simplify language, avoid legal risks, and support language learning. Parents get a preview, and kids learn vocabulary and values through playful stories.
Prompt Structure Overview:
✅
Goal: What the model should do.
🔹
Return Format: What to return and how to structure it.
⚠️
Guardrails: Avoids errors, sets legal and safety boundaries.
🚫
IP Safety: Prevents use of copyrighted or branded content.
🖌️
Storytelling Principles: Guides story flow (Character → Problem → Emotion → Resolution).
📄
Context Dump: Explains the prompt’s purpose and audience.
Prompt Implementation
Prompt Chaining for More Accurate Output
Working with the UX Engineer, the original single prompt worked in GPT playground but didn’t perform well in the app. We split it into four prompts to improve accuracy, handle inputs better, and control the output flow. We later switched to the Gemini 2.0 model due to better cost-effectiveness.
Designing the App That Makes AI Feel Magical
Overview of App
Quick Demo of Prototype
I collaborated with UX engineer to map out a system diagram. This clarified what the prompt needed to extract from the JSON file and at which step of the user flow that data would be used.
To minimize the effort of story creation for busy parents, we structured the app into three simple sections: Onboarding, Preview, and Story Mode and Library.
Onboarding: Helps parents start quickly by guiding through language selection, reading level and toy upload.
Preview: Gives a quick look at the story setup, allowing parents to make small adjustments.
Story Mode: Final output for storytelling with their child.
Library: Stores previously scanned toys and generated stories for easy reuse and quicker access.
Here's the Process
Usability Test
Seeing the Magic Firsthand Through Testing
The team conducted 3 rounds of testing with bilingual parents and their children, observing how they used the app. We focused on engagement, learning value, and where they wanted more control.
During testing, we saw the power of design when the child lit up with joy as she experienced her toy coming to life in the story.
Image displayed with prior consent from parent/guardian.
Feedbacks & Iterations 1: Cultural Sensitivity
🌎 Flags Didn’t Feel Inclusive for Multilingual or Diaspora Communities.
Initially, I used flags to reduce cognitive load by providing familiar visual cues, especially for parents scanning quickly or with limited literacy.
Iteration: Replaced flags with native language labels (e.g., “Español”, “中文”) to shift focus from national identity to language itself, and changed "Others" to "More Languages"
Why: This promotes cultural sensitivity, avoids political assumptions, and feels more inclusive for multilingual and diaspora communities.
✅ Benefits of the Change
More Inclusive: Avoids political associations and better reflects global, multilingual communities
Culturally Sensitive: Doesn’t favor one region for widely spoken languages.
Focuses on the language, not the flag.
⚠️ Trade-offs
Less Instant Visual Recognition: Flags are faster to identify, especially for unfamiliar scripts.
Slightly Higher Effort: Some parents may take longer to read native labels.
But with only 5 options, this has little impact
Feedbacks & Iterations 2: Personalized Vocabulary Learning
📚 Parents Loved the ‘Words to Learn’ Section, but Wished They Could Customize.
Iteration: Added the option for parents to swap out vocabulary words during story preview, letting them replace words their child already knows.
Why: This feature gives parents more control, personalizes the learning experience, and supports different developmental stages, especially valuable in bilingual homes.
To get to this, I explored these 2 designs:
Exploration 1: “Re-Choose” Button
✅ Lower effort, supports quicker decisions.
✅ Can use reading level from onboarding to improve accuracy.
⚠️ Less predictable, parents may not trust the output.
Exploration 2: Select Words Directly from the Story
✅ Gives parents full control to choose relevant words.
⚠️ Increases cognitive load, requires reading the entire story first.
Feedbacks & Iterations 3: Enhancing Story Engagement
🧸 Let More Toys Come Together to Make the Stories More Dynamic
Iteration: Expanded story generation to include multiple toy characters for group adventures.
Why: Supports richer storytelling, encourages social behaviors like teamwork, and reflects how children naturally play with multiple toys.
Testimonials & Accomplishments
The AI-powered storytelling concept brought unexpected joy to families and sparked strong emotional engagement.
✅ Emotional resonance: Parents described the experience as “magical” and appreciated how it relieved the pressure to teach.
✅ Delight & adoption signals: At our public exhibition, visitors said:
“It feels like the toy is talking to my child.”
“I would definitely use this. My nephew would love it.”
“Looking forward to seeing it on the market.”
These proved strong potential for turning language learning for children into joyful, everyday play.
Photos were shared with participants’ permission.
Concerns in Using AI for Storytelling
🧠
Loss of Cultural and Emotional Nuance
AI-generated stories may miss the depth, humor, and traditions that make heritage storytelling meaningful.
Consideration: Can AI capture dialects and storytelling styles that resonate emotionally across cultures?
🤖
Over-Reliance on AI for Language Learning
While AI can assist, it risks displacing meaningful parent-child interactions if overused.
Consideration: How do we design AI tools to enhance, not replace, human connection?
🎯
Accuracy and Bias in AI-Generated Content
Training data often reflects dominant cultural norms, leading to oversimplified grammar or inaccurate cultural representations.
Consideration: How can we introduce parent oversight and cultural context without overwhelming the user?
Continuing the Story…
📊
Data Tracking
Embed data tracking to measure early engagement and learning impact, focusing on:
Story completion rate.
User retention (Day 1, 7, 30).
Number of stories generated / user
📔
Diary Study Testing
Conduct a 2–4 week diary study with bilingual families using the prototype, to gather insights on:
Effectiveness on language learning: How often the child interacts with their mother tongue and do they remember the vocabulary.
Child Engagement Over Time: Whether initial excitement fades or deepens.
📱
UXUI Refinement
Iterate on story creation flow, visuals, and voice tone based on feedback, to improve usability, engagement, and ensure the experience is intuitive for both children and parents.
Utilizing AI in a Meaningful Way
Not Just Designing Prompts, But Designing Prompt Strategy
The order and structure of prompts affect how the AI answers. Breaking complex prompts, chaining, and few-shot learning improved story flow, vocabulary, and accuracy.
Balancing AI Creativity with User Control to Keep Stories Personal and Easy
Effective AI storytelling balances creative automation with parent input. Too much automation feels impersonal, and too much manual effort creates friction.
When You Design for Kids You Also Design for Parents
Though designed for children, parents guide the experience. The design must address both the parent’s needs and the child’s learning experience, ensuring both find value and ease in the interaction.
Meet the Humans Behind the Magic
Powered by collaboration, curiosity, and a whole lot of stuffed animals.
Next Project
Rethink The Way We Invest
Empower Investors to Invest With Confidence
In response to the gap between traditional platforms and next-generation investors, I was tasked by Good Finance Securities to redesign the stock trading app’s order page for a more intuitive experience that meets today's investment needs.
