Skip to main content

​
Shell Integration

Shells are platform-specific hosts that embed Soul Kernel, providing native experiences on iOS, Unity, Physical AI, and more.

​
Overview

A Shell:
  • Embeds the Soul Kernel binary
  • Provides platform-specific UI/UX
  • Handles device-specific capabilities
  • Manages lifecycle and resources

​
Architecture

​
Supported Shells

​
1. iOS Shell (React Native)

Purpose: Mobile companion app for iPhone/iPad Features:
  • Native iOS UI with React Native
  • Camera, microphone, haptics integration
  • Background sync and notifications
  • App Store distribution ready
Integration: 
// Swift bridge to Rust kernel
@objc class SoulBridge: NSObject {
    private let kernel: OpaquePointer
    
    init() {
        kernel = soul_kernel_init()
    }
    
    @objc func ask(_ query: String) -> String {
        return String(cString: soul_kernel_ask(kernel, query))
    }
}

​
2. Unity Shell

Purpose: Game NPCs and AR/VR experiences Features:
  • C# bindings via gRPC
  • Unity Sentis for on-device inference
  • Animation and physics integration
  • Cross-platform (PC, Mobile, VR)
Integration: 
// Unity C# client
public class SoulClient : MonoBehaviour {
    private GrpcChannel channel;
    private SoulService.SoulServiceClient client;
    
    void Start() {
        channel = GrpcChannel.ForAddress("localhost:50051");
        client = new SoulService.SoulServiceClient(channel);
    }
    
    async Task<string> Ask(string query) {
        var response = await client.AskAsync(new AskRequest { Query = query });
        return response.Answer;
    }
}

​
3. Physical AI Shell

Purpose: Robotics and IoT devices Features:
  • Runs on edge devices (Jetson, RPi)
  • Sensor/actuator integration via dora-rs
  • Real-time processing capabilities
  • OTA updates
Integration: 
// dora-rs node for physical AI
#[dora::node]
async fn physical_ai_node() {
    let soul = SoulKernel::new().await?;
    
    // Subscribe to sensor events
    dora::subscribe("camera/frame", |frame| {
        let result = soul.process_vision(frame).await?;
        dora::publish("motor/command", result.action);
    });
}

​
4. Web Shell (Planned)

Purpose: Browser-based Soul experiences Features:
  • WASM compilation of kernel
  • WebRTC for voice/video
  • PWA capabilities
  • Cross-browser support

​
Creating a Custom Shell

​
1. Define Requirements

shell:
  name: "custom-shell"
  platform: "linux"
  capabilities:
    - audio_input
    - audio_output
    - display
  kernel_version: ">=0.1.0"

​
2. Implement Shell Interface

pub trait Shell {
    // Lifecycle
    fn initialize(&mut self, config: ShellConfig) -> Result<()>;
    fn shutdown(&mut self) -> Result<()>;
    
    // Communication
    fn handle_input(&mut self, input: UserInput) -> Result<()>;
    fn render_output(&mut self, output: SoulOutput) -> Result<()>;
    
    // Platform-specific
    fn get_capabilities(&self) -> Vec<Capability>;
}

​
3. Platform Integration

Each shell must handle:
  1. Input Sources
    • Touch, keyboard, voice
    • Sensors, cameras
    • Network events
  2. Output Rendering
    • Text, speech, visuals
    • Haptics, actuators
    • Notifications
  3. Resource Management
    • Memory constraints
    • Battery optimization
    • Network usage

​
Shell Communication Patterns

​
1. Direct Embedding (iOS, Physical AI)

// Kernel runs in-process
let kernel = SoulKernel::new()?;
let response = kernel.ask("Hello").await?;

​
2. IPC/gRPC (Unity, Desktop)

service SoulService {
    rpc Ask(AskRequest) returns (AskResponse);
    rpc Remember(RememberRequest) returns (RememberResponse);
    rpc ExecuteSkill(SkillRequest) returns (stream SkillResponse);
}

​
3. Message Queue (Distributed)

// Pub/sub pattern for scalability
soul.subscribe("user_input", handle_input);
soul.publish("soul_response", response);

​
Best Practices

  1. Minimize Latency - Keep kernel close to UI
  2. Handle Offline - Graceful degradation
  3. Respect Platform - Follow native conventions
  4. Optimize Resources - Battery, memory, network
  5. Test on Device - Not just simulators

​
Platform-Specific Considerations

​
iOS

  • App Transport Security for sync
  • Background processing limits
  • Privacy permissions (camera, mic)

​
Unity

  • Render thread vs logic thread
  • Platform-specific plugins
  • Build size optimization

​
Physical AI

  • Real-time constraints
  • Hardware abstraction
  • Safety mechanisms

​
Debugging Shells

​
Logging

// Enable shell-specific logging
RUST_LOG=soul_kernel::shell=debug cargo run

​
Profiling

# CPU profiling
perf record --call-graph=dwarf ./soul-shell
perf report

# Memory profiling
valgrind --leak-check=full ./soul-shell

​
Testing

#[test]
fn test_shell_lifecycle() {
    let mut shell = MockShell::new();
    shell.initialize(config).unwrap();
    
    // Test input handling
    shell.handle_input(UserInput::Text("Hello")).unwrap();
    
    // Verify output
    assert_eq!(shell.last_output(), "Hello! I'm your Soul.");
}

​
Next Steps