Students Immerse In Deep Learning with XR Experiences

Collaborating With Stakeholders To Meet Desired Outcomes.

Agency: Strive9   Client: Joy Media   Duration: 2 months

Joy Media, came to me with a mission to redefine the way students interact with history and culture.

In their research they discovered an opportunity

to help students who don't have access (lack of funding, location, etc.) to historical and cultural experiences provided by the great museums found around the world.

Joy Media's founder not only wanted to provide an educational lift to these students but also knew that by immersing in different cultures students develop a strong sense of empathy and understanding of others.

We decided that by creating an interactive experience using Augmented Reality (AR) and Virtual Reality (VR), we could make history and culture more accessible and engaging for students in any learning environment.

As the lead Product Designer, my role was to:

  • Become a domain expert in designing for 3D digital spaces
  • Collaborate with engineers to define practical, scalable components
  • Understand the needs of museums and students

The desired outcomes that defined success for this product were it's abilities to generate demand from museums (buyer) and drive engagement with students (target audience).

Our project had a tight deadline—an MVP needed to be ready within two months for a conference, where the founder aimed to pitch the MVP to museums like the Smithsonian.

Transforming Artifacts into Narratives

Crafting Intuitive Navigation for AR and VR Experiences

Designing navigation and interactions for the AR mobile app required a strategic approach to balance usability, immersion, and accessibility within limited screen space. I carefully positioned UI elements to avoid obstructing critical visual information while keeping them easily accessible. To achieve this, I leveraged spatial anchors, contextual UI, and gesture-based interactions, minimizing clutter and enhancing the user experience.

In contrast, the VR desktop experience offered a more expansive virtual environment, allowing for spatial and immersive navigation. To prevent information overload, I kept most interactions within a hidden menu, ensuring users could fully immerse themselves in the digital space without unnecessary distractions.

Implementing Gen AI in 3D Experience

The founder and lead engineer envisioned Griot Mode, a generative AI-powered experience designed to facilitate seamless, interactive storytelling through voice and text-based conversations. The goal was to ensure a fluid and consistent user journey across both AR and VR environments.

We introduced Griot Mode as a dynamic digital storyteller, enabling real-time, two-way interactions. Users could engage with the experience by asking questions and uncovering rich narratives tailored to each artifact. Whether exploring a Yoruba mask or an ancient drum, Griot Mode provided a culturally informed, adaptive experience that deepened users’ connection to history and heritage.

Getting Creative within a 3D world

Exploring creativity in a 3D world meant transporting users to the very settings where artifacts originated. With a single click, they could zoom in on intricate details, uncovering the unique features of each piece. Face recognition, object placement, and interactive engagements were just a few of the innovative techniques we leveraged to showcase the platform’s potential. These dynamic interactions not only enriched the user experience but also demonstrated the limitless possibilities to our future partners.

User's Journey

Designing the user journey required careful consideration—from the moment they discovered the app to their initiation of the AR or VR experience. Anticipating user behavior, we collaborated closely with stakeholders to create seamless entry points. We determined that the AR mobile experience would begin with a QR code interaction on flyers, pamphlets, and web catalogs, ensuring easy access. Meanwhile, the VR experience would initially be anchored to the desktop, providing a more immersive yet accessible introduction to the platform.

A Blank Canvas in Unfamiliar Territory

Designing for AR and XR was uncharted territory for me, requiring deep research

into best practices, technological possibilities, and the unique constraints of immersive environments.

Unlike traditional interfaces, these experiences had to feel intuitive in 3D space while maintaining consistency across AR and VR interactions. Ensuring that users could seamlessly engage with Griot Mode—whether through voice, text, or object interactions—added another layer of complexity.

One of the most significant challenges was balancing the needs of end users with the priorities of investors and partners.

While the experience was ultimately designed for students, early-stage startup realities meant that the prototype had to demonstrate value to stakeholders first. This led to design decisions that prioritized marketability and funding potential, sometimes requiring trade-offs between deep educational engagement and high-impact features that impressed potential partners.

We also had to strategically map the user journey, from pre-app discovery to initiating the AR or VR experience.

Anticipating where users would first interact with the platform was crucial. Through collaboration with stakeholders, we determined that QR codes on flyers, pamphlets, and web catalogs would serve as the entry point for the AR mobile experience, while the VR experience would remain desktop-based—a limitation dictated by existing technology and accessibility concerns.

Feedback from Joy Media’s Engineering Lead and CEO reinforced the need for a professional, consistent, and purpose-driven interface.

This meant striking a delicate balance: ensuring the design reflected the platform’s cultural depth and educational value while maintaining the polish and coherence required for a compelling product. Showcasing features like face recognition, object placement, and real-time interactions helped highlight the potential of the technology to future partners while still keeping the user experience engaging and meaningful.

Ultimately, these challenges pushed me to think holistically about immersive design, stakeholder alignment, and user experience strategy, ensuring that the designs were both marketable and impactful.

Creating New Avenues for Educational Storytelling

The XR Museum platform successfully demonstrated how immersive technologies could transform museum experiences,

making cultural storytelling more accessible, engaging, and scalable. By integrating AR and VR, we unlocked new ways for audiences to interact with artifacts, deepening their connection to history and heritage.

Key outcomes included:

  • Increased Accessibility: XR experiences enabled users—regardless of location—to explore museum exhibits, making cultural heritage more inclusive and available to a wider audience.
  • Enhanced Engagement: Interactive storytelling provided a richer experience, allowing visitors to go beyond static descriptions and instead engage in dynamic, real-time conversations with artifacts.
  • Scalability Across Institutions: The platform’s modular design allowed for customization and branding, making it adaptable to the unique needs of different museums, educational institutions, and cultural organizations.

‍
By bridging technology and storytelling,

the XR Museum not only showcased the potential of Griot Mode but also laid the foundation for future partnerships and expanded applications in the world of immersive cultural experiences.

What's Next?

This proof of concept laid the foundation for future innovations, demonstrating the potential of Griot Mode as a powerful tool for immersive storytelling. The project not only validated the feasibility of AR and XR experiences in museums but also revealed exciting opportunities for expansion.

  • Educational Personalization – Enhancing Griot Mode to offer tailored learning experiences, allowing visitors to explore artifacts based on their individual interests and knowledge levels.
  • Museum Collaborations – Connecting collections across institutions, enabling cross-museum storytelling where artifacts from different locations come together to form a cohesive narrative.
  • Global Accessibility – Expanding beyond mobile and desktop by integrating XR experiences into streaming platforms like Roku and Fire Stick, bringing immersive learning to a wider audience.

Key Takeaways

  • Purposeful Design – Thoughtfully crafted interactions balanced 2D and 3D elements, ensuring both usability and cultural sensitivity in an immersive setting.
  • Story-Driven Approach – The project emphasized narrative connectivity, ensuring that artifacts weren’t just displayed but woven into a larger, meaningful story.
  • Scalable & Adaptable – A modular, customizable platform allowed for seamless integration across multiple museums, setting the stage for broader adoption and future applications.

This project was just the beginning. With these insights and learnings, the next steps involve refining the experience, forging new partnerships, and pushing the boundaries of XR-powered cultural storytelling.

Thanks to -

Sven - Lead Engineer

Falona - Founder/CEO

Brought to you by Dylan Rose

Have a great day!