A sandy trail winds through coastal dunes under a cloudy sky. Three people stand near a wooden signpost that reads 'Noordduinen – Gele wandelroute' with an arrow; one person holds a small dog on a leash. Tall grasses cover the rolling dunes, and a metal transmission tower rises in the distance. A long strap or lead lies on the sand in the foreground, adding to the casual, mid-hike feel of the scene.

Phase 2 Report

Screen-to-Soundscape (STS) evolved from a speculative concept into a working, downloadable prototype that lets people explore cities by ear. In Phase 2, our goal was to prove that open geographic data (OpenStreetMap) and spatial audio could create a learnable, navigable sound world—prioritizing exploration over step-by-step routing.

Key Deliverables

  • Godot-based desktop application
  • Auto-starting tutorial
  • Ambient audio "beds"
  • Symbolic point-of-interest icons
  • Surface-aware footsteps
  • Continuous world loading (Netherlands and Belgium)
  • Location-aware AI assistant

Phase 2 Methodology

The team employed "short build → test → learn cycles with blind and visually impaired co-creators as decision-makers." Key priorities included:

  • Keyboard-first interaction design
  • Restartable first-run tutorial
  • Audio grammar mapped to OpenStreetMap tags

Phase 2 Co-Creation

A small co-creation/work session in a bright room. Three participants wearing headphones work on laptops at a long table while another person stands and points, offering guidance. Mugs, a water carafe, and notes on a corkboard set an informal, collaborative atmosphere.

Co-creation workshop session with participants testing the prototype

Workshop Summary: "We ran two co-creation workshops and one listening party. Sessions covered device habits (screen readers, headphones), exploration tasks (finding a toilet, following a busy street), and comfort (fatigue, silence)."

Key outcomes included co-written tutorials, refined sound density adjustments, added borders/street "walls," and prioritization of "typed commands after testers noted typing is often faster and more reliable than speech across languages."

Prototype Development and Testing

Initial prototype used A-Frame web framework on Glitch. First co-creation feedback identified need for auditory cues and boundary indicators. Second iteration added:

  • Audio boundaries
  • Additional keyboard controls
  • Refined sound parameters
  • Adjusted distance modeling

Phase 2 Results

Stack & Hosting

  • Local Godot application
  • OpenStreetMap/Overpass queries
  • DigitalOcean droplet hosting (maintained through December 2025)
  • Offline city bundle support
  • Fallback local summary when AI unavailable

Sound Model Components

  • Distance attenuation and stereo/HRTF panning
  • Gentle occlusion
  • Constant ambient floor
  • Surface-aware footsteps (asphalt, cobble, gravel, grass, wood, polished floor)
  • Symbolic POI icons
  • System state earcons

Tutorial Features

  • Auto-starts on first run
  • Toggle with H key
  • Advance with Tab
  • Restart with F1
  • Conditional prompts for wall interaction
  • Distinct state sounds

Unexpected Outcomes

Three significant discoveries:

  1. "Tutorial became scaffolding for independence—not just instructions"
  2. Constant ambient floor materially reduced anxiety
  3. Symbolic sounds learned instantly and retained

Limitations & Pending Work

Challenges identified:

  • Cognitive load management
  • Need for level-of-detail and scale presets
  • Late-arriving state cues
  • Alt-alt-text computer-vision layer (prototype stage, not production-ready)
  • Global scale out of scope; NL/BE deliberate for performance

Phase 2 Outputs/Tools

Software

  • Title: Screen-to-Soundscape (Prototype Software, NL/BE)
  • Launch: September 2025
  • Public demo: 26 September 2025
  • GitHub: Version 1.0.5

Workshops

  • Co-creation Workshop #1: February 2025
  • Co-creation Workshop #2: May 2025
  • Listening Party/Public Demo: 26 September 2025 (~10 participants)

The Future

Next Steps

Finish alt-alt-text interface, enhance spatial audio (echoic footsteps, clearer spawn/command states, stronger borders), expand detail/scale presets, add time-of-day and city-specific ambiences, support community sound packs, and ship offline bundles. Longitudinal studies planned.

Sharing Plans

Website maintenance, GitHub releases, technical documentation, demo video with transcripts, and proposed conference workshops.

Funding

"We do not have follow-up funding secured. We seek €50,000–€100,000 to deliver a fully functional application, pay co-creators, complete alt-alt-text, deepen spatial audio, package offline bundles, and run studies."

Exploration of accessibility/digital-culture grants, civic partnerships, and sponsorships while maintaining open-source core stack. Phased city-bundle releases if partial funding secured (Leiden/Leuven, Amsterdam, Brussels priority).