Skip to main content

Shell Integration

Shells are platform-specific hosts that embed Soul Kernel, providing native experiences on iOS, Unity, Physical AI, and more.

Overview

A Shell:
  • Embeds the Soul Kernel binary
  • Provides platform-specific UI/UX
  • Handles device-specific capabilities
  • Manages lifecycle and resources

Architecture

Supported Shells

1. iOS Shell (React Native)

Purpose: Mobile companion app for iPhone/iPad Features:
  • Native iOS UI with React Native
  • Camera, microphone, haptics integration
  • Background sync and notifications
  • App Store distribution ready
Integration:

2. Unity Shell

Purpose: Game NPCs and AR/VR experiences Features:
  • C# bindings via gRPC
  • Unity Sentis for on-device inference
  • Animation and physics integration
  • Cross-platform (PC, Mobile, VR)
Integration:

3. Physical AI Shell

Purpose: Robotics and IoT devices Features:
  • Runs on edge devices (Jetson, RPi)
  • Sensor/actuator integration via dora-rs
  • Real-time processing capabilities
  • OTA updates
Integration:

4. Web Shell (Planned)

Purpose: Browser-based Soul experiences Features:
  • WASM compilation of kernel
  • WebRTC for voice/video
  • PWA capabilities
  • Cross-browser support

Creating a Custom Shell

1. Define Requirements

2. Implement Shell Interface

3. Platform Integration

Each shell must handle:
  1. Input Sources
    • Touch, keyboard, voice
    • Sensors, cameras
    • Network events
  2. Output Rendering
    • Text, speech, visuals
    • Haptics, actuators
    • Notifications
  3. Resource Management
    • Memory constraints
    • Battery optimization
    • Network usage

Shell Communication Patterns

1. Direct Embedding (iOS, Physical AI)

2. IPC/gRPC (Unity, Desktop)

3. Message Queue (Distributed)

Best Practices

  1. Minimize Latency - Keep kernel close to UI
  2. Handle Offline - Graceful degradation
  3. Respect Platform - Follow native conventions
  4. Optimize Resources - Battery, memory, network
  5. Test on Device - Not just simulators

Platform-Specific Considerations

iOS

  • App Transport Security for sync
  • Background processing limits
  • Privacy permissions (camera, mic)

Unity

  • Render thread vs logic thread
  • Platform-specific plugins
  • Build size optimization

Physical AI

  • Real-time constraints
  • Hardware abstraction
  • Safety mechanisms

Debugging Shells

Logging

Profiling

Testing

Next Steps