Hieronymus Bosch, The Garden of Earthly Delights, oil on oak panels, 205.5 cm × 384.9 cm (81 in × 152 in), Museo del Prado, Madrid
Prototype
Current Prototype
The executable is available via GitHub at version 1.0.5. The tutorial launches automatically on first run.
Operational Model
The application functions as a local (Godot) desktop prototype that typically connects to a DigitalOcean server for map data and AI assistance. Offline functionality is supported for demonstrations, with local handling of movement, spatial audio, and tutorials.
Core Experience
Users explore urban environments through audio rather than visuals. The system provides ambient soundscapes, auditory icons for nearby locations, and surface-responsive footsteps. Navigation uses arrow keys with periodic AI assistant access.
Technical Architecture
The system integrates OpenStreetMap data with spatial audio processing, covering Netherlands and Belgium regions. Voice synthesis and AI models run server-side to optimize client performance.
Prototype Development
Initial iterations addressed navigation challenges identified through co-creation sessions. Developers implemented boundary audio cues and keyboard controls to enhance user orientation and control.
Design of first prototype
The initial prototype was built using the A-Frame framework, hosted on Glitch. This web-based prototype featured keyboard navigation and audio triggers at spatial points.
Feedback from first co-creation
Users appreciated the spatial layout but needed stronger auditory boundary indicators. Key findings included the need for clearer navigation cues and better orientation feedback.
Reflection on the first co-creation
The team learned that traditional screen readers often "flatten" web experiences by reducing content to linear lists, eliminating spatial context crucial for understanding complex information like maps or images. This insight drove the development of enhanced spatial audio features.
Design of the second prototype
The second prototype added enhanced audio boundaries, refined sound parameters, and adjusted distance modeling. Additional keyboard controls were implemented to give users more agency in navigation.
Feedback from the second co-creation
The refined prototype received positive feedback for its improved boundary audio and control options. Users noted that the enhanced sound design made navigation more intuitive and less disorienting.
Reflection on the second co-creation
Co-creators valued control over voice characteristics, sound localization, and movement within soundscapes. While exploration appealed to participants, they highlighted difficulties navigating without clear auditory cues and complexity from multiple layered voices.
Alt-Text Generation Examples
Using AI-powered image analysis, Screen-to-Soundscape can generate customized alt-text descriptions tailored to different audiences and contexts. Below are examples using Hieronymus Bosch's "The Garden of Earthly Delights":
Garden of Earthly Delights - Custom Alt-Text for Art Curator
Detailed art historical description for an art curator perspective
Garden of Earthly Delights - Custom Alt-Text for a Child
Child-friendly description with simpler language
Garden of Earthly Delights - Custom Alt-Text for a Child (Upbeat tone)
Child-friendly description with an upbeat, enthusiastic tone
Garden of Earthly Delights - Custom Alt-Text for a Child (Upbeat tone and Soundscape)
Child-friendly description with upbeat tone and immersive soundscape
Plan for the future co-creation
Future development will focus on:
- Expanding co-creation sessions with diverse visual content (charts, infographics, complex materials)
- Promoting open-source participation from developers, sound designers, and accessibility advocates
- Documenting co-creation guidelines for future inclusive design projects
- Enhancing spatial audio with echoic footsteps and clearer state transitions
- Supporting community sound packs and offline city bundles