Sound Research

The project aims to create auditory experiences for blind and visually impaired users, with sound cues that would guide users through the digital landscape.

Developing Sound Cues

Creating sounds that function simultaneously without overwhelming users requires careful attention to timing and layering to support information flow. The challenge is to design an auditory environment that is both rich and navigable.

Integrating Theories of Space and Scale

The sound design references Daniel R. Montello's "Scale and Multiple Psychologies of Space," implementing four spatial categories:

  • Immediate (Figural) Space: Quick, concise cues for static webpage information
  • Vista Space: Moderately prolonged sounds conveying layout and arrangement
  • Environmental Space: Expansive soundscapes evolving through navigation
  • Geographical Space: Complex, layered sounds for extensive information sections

Creating Harmonious Soundscape

Tones, pitches, and rhythms work cohesively as navigational aids, creating a soundscape that is both informative and aesthetically pleasing. The goal is to transform two-dimensional web content into a rich, four-dimensional auditory experience.

User Feedback Adaptation

The sound design incorporates boundary sounds and iterative refinement through co-creation sessions. A key insight from users: "silence feels like being lost" – emphasizing the importance of constant ambient audio presence.

Sound Examples

Below are examples of the sounds used in the Screen-to-Soundscape prototype. These audio files demonstrate the various sound categories and their implementation in the system.

Surface Sounds

Materials include grass, concrete, asphalt, wood, gravel, and water. Footsteps indicate ground type, providing tactile feedback through audio.

1. Surface: Asphalt

Footstep sounds on asphalt surface

2. Surface: Paved

Footstep sounds on paved surface

3. Surface: Unpaved

Footstep sounds on unpaved surface

Feature Sounds

Immediate icons—scissors (hairdresser), school bell, leafy rustle (trees)—identify specific locations and help users understand their immediate environment.

4. Highway - Tertiary

Sound of a tertiary highway

5. Highway - Crossing

Sound of a highway crossing

6. Highway - Bus Stop

Sound of a bus stop

Symbol Sounds

Atmospheric elements—highway whoosh, park birds, station PA, door motifs—convey space and activity level, creating a sense of place.

7. Landuse Farmland (Atmosphere)

Atmospheric sound of farmland

8. Landuse Farmland (Symbol)

Symbolic sound representing farmland

9. Amenity - School (Atmosphere)

Atmospheric sound of a school

10. Amenity - School (Symbol)

Symbolic sound representing a school (bell)

11. Natural - Water

Natural water sound

12. Natural - Wood

Natural wood sound (footsteps on wood)

13. Landuse - Grass

Grass footstep sound

Phase 2 Sounds

The focused sound set includes:

Ambient Beds

  • Traffic hush
  • Canal water
  • Park birds and wind
  • Rail hum
  • Market murmur with distance attenuation

Symbolic Auditory Icons

  • Doorbell (shops)
  • Revolving door (malls/offices)
  • Scissors (hairdresser)
  • Freewheel click (bike shops)

Public Transport Motifs

  • Bus stop ping
  • Tram ring
  • Station PA blur

Interaction Earcons

  • Spawn chime
  • Command-mode blip
  • Loading cues
  • Teleport whooshes
  • Orientation pips
  • Wall feedback

Sound Categories

Interaction Sounds

System feedback for AI listening/replying, boundary collisions, landing, command mode, loading, zoom, and teleport states. These sounds help users understand system state and interaction results.

Credits

Primary Sound Designer: Colette Aliman

Contributors

  • Florian Reichelt (@florianreichelt)
  • DWOBoyle, Yle (Finnish Broadcasting Archives), User 150025, suntemple, SkullkSound, FxProSound, spycrah, Suso Ramallo, Valparaíso Inaudible, Geenburg, GnomeKid, plantmonkey (freesound.org)
  • The Dutch Sound Museum