PROJECTION HOLOGRAM & AI AVATAR AGENT PAHLAWAN
Interactive Holographic Hero Avatar Prototype
ABOUT THE PRODUCTION
A prototype development of an interactive holographic experience featuring an AI-powered historical hero avatar. The prototype combines a 45-degree glass reflection optical technique (Pepper’s Ghost) with an AI conversational agent system, allowing museum and exhibition visitors to converse naturally with a lifelike virtual historical hero. Focused on developing the core Unity application, integrating and fine-tuning the conversational AI engine with domain-specific knowledge, and synchronizing real-time vocal responses and facial blendshape lip-syncing.
TECHNOLOGY & RIG
Conversational LLM Engine
Pepper’s Ghost Holographic Illusion
Research & Development
Unity & Conversational AI
PROTOTYPE BENCHMARKS & ARCHITECTURAL HIGHLIGHTS
Pepper’s Ghost optical reflection creating a floating 3D avatar without wearable headsets.
Fine-tuned LLM system prompt embodying historical knowledge and national hero persona.
Real-time facial blendshape deformation synchronized with synthesized speech phonemes.
Fully conversational vocal pipeline: speech capture, AI reasoning, voice reply, and gesture response.
01 // PROTOTYPE OVERVIEW & TECHNICAL FOCUS
Projection Hologram & AI Avatar Agent Pahlawan
Interactive Holographic Hero Avatar Prototype · Surabaya, Indonesia
A prototype development of an interactive holographic installation featuring an AI-powered Indonesian historical hero avatar. The system merges classical 45-degree glass reflection optics (Pepper’s Ghost) with an intelligent conversational agent, allowing museum and exhibition visitors to converse naturally with a lifelike virtual historical figure floating in physical space.
Fine-tuned with dedicated system prompts and guardrails, allowing visitors to ask about revolutionary struggles, historical battle dates, and educational philosophies.
02 // CORE CONTRIBUTIONS & SYSTEM INTEGRATION
Unity-Based Interactive Hologram Prototype
Architected and built the core real-time Unity 3D application, managing scene states, lighting, high-contrast black backgrounds, and avatar rendering.
45-Degree Glass Reflection Technique
Engineered the physical optical chamber, aligning monitor brightness, reflection angles, and ambient light shielding to maximize holographic realism.
AI Conversational System Integration & Fine-Tuning
Integrated Large Language Model (LLM) APIs, crafting specialized system prompts and knowledge guardrails to authentically embody a historical hero persona.
Real-Time Avatar & Visitor Interaction
Engineered smooth transitions between idle breathing, active listening, thinking indicators, and expressive speech delivery with audio-driven lip-sync.
Unified Multimodal Prototype Pipeline
Consolidated audio input capture, cloud speech recognition, AI reasoning, voice synthesis, and visual rendering into a unified, responsive loop.
03 // MULTIMODAL AI & HOLOGRAPHIC PIPELINE
Acoustic Input & Transcription
Captures visitor spoken queries in Indonesian through directional microphones, transcribing speech into digital text tokens with background noise filtering.
Conversational Agent Reasoning
The conversational engine analyzes visitor queries within historical context, generating historically accurate, inspiring, and persona-aligned responses in real time.
Voice Synthesis & Blendshape Lip-Sync
Converts textual output into an expressive, authoritative voice timbre while calculating real-time audio amplitudes to drive avatar mouth visemes and facial blendshapes.
45° Optical Hologram Projection
The Unity camera renders the illuminated avatar on a pitch-black background, which reflects onto a high-transparency 45° glass plate, creating a volumetric illusion of a floating character.
04 // VISUAL DOCUMENTATION REPOSITORY
public/images/projects/hologram-ai-avatar/
Recommended photos: 01-hologram-prototype (glass reflection view), 02-display-setup (45° glass angle & monitor chamber), 03-ai-avatar-interaction (character model & dialogue), 04-unity-ai-development (Unity & LLM setup), 05-prototype-demonstration.