Home > Agorà Immersive Room

Interactive 3D Viewer

You can use your mouse or touch screen to rotate, pan, and zoom into the 3D space.

Design of the structure

The idea was to build an Ambisonics speaker array with 3 rings, each with 8 speakers, plus a 25th speaker at the azimuth, directly above the center.

This was supposed to be installed in a specific room in the municipality's music school in Faenza. The structure was designed to maximise the radius of the speaker array within the physical constraints of the room.

This resulted in a 3m radius, vertically offset from the floor. The structure was simply drawn in Blender, using the 3D scan for reference, and an icosphere for setting identical distances to each speaker.

The resulting structure is an octagon with sides of 2.5m, a diameter of 6m, and a height of 3.8m. A further horizontal ring sits 1m below the top. Originally, this was meant to be the base of a structure that allowed the upper ring to be closer to the listener. Similarly, the lower speaker ring was meant to be mounted directly on the floor. The bulk of the structure is made of 40mm Bosch Rexroth profiles, and weighs 160kg. The limited weight and the length of the pieces make the system somewhat transportable even with a regular car.

During the course of the project, though, the installation was delayed, mostly due to idiotic procurement rules. We were supposed to buy everything in a lump transaction, for which we had to get three competing offers. Since the products needed for this system belong to very different categories, this was very difficult to arrange. By the time this was handled, the room in Faenza was not available anymore. The municipality of Tresigallo and the Torri di Marmo association stepped in, and made the Sogni building available for six months.

Built as the public baths of Tresigallo, it was rebranded as Sogni (dreams) after a restoration. Even in the low season, there is a surprising number of tourists stopping for identical photos of the façade. the whole town was build in the "rationalist" style during the fascist dictatorship. This room matches the available dimensions of the structure quite nicely, but as it is an historic building, it was not possible to drill the anchoring points as originally designed. This required a whole lot of work in stabilizing the structure. The building suffers terribly from greenhouse effects, and as it is a concrete box, it is really quite reverberant. Currently, it also has sealing issues (during rainstorms, the windows don't hold) and electrical issues (the grounding is so bad that initially plugging in the HDMI cables resulted in sparks, and the lights have dodgy relays, the only way of actually turning them off is from the mains panel).

In order to keep the structure adaptable to evolving circumstances, I had bought variable angle joints for the horizontal elements, instead of the 45° angle ones.

This resulted in two problems: first, I forgot to specify the 17mm holes in the pieces, preventing installation; second, the encumberance of this mounting solution prevented us from installing all of the elements that were supposed to bind the horizontal elements to the vertical columns. The first issue was solved by my friend Marco, who used his own company's resources and machinery to drill the required holes. That took around 10 days, but it worked perfectly.

At first, it was really quite wobbly, which required some creative thinking, and many consultations with architects, who also helped out of sheer friendship. This included coming all the way to Tresigallo, during their holidays, to help with the installation.

After several hundred Euros in twisted steel cable, connectors, and copious amounts of metal joinery, we finally succedded in assembling the structure; the end result only shows the flex of the aluminum itself, and really seems quite solid.

The main interventions were the installation of several steel cable "x" shapes, making the affected sides more rigid. There is also an "x" at the top. The problem is that as the "feet" of the columns could not be bolted to the floor, excessive tensioning on the top "x" would destroy the structure, dragging the top of the columns towards the center. Each speaker was simply bolted to the structure using an OEM wall mount. In this way, they're not equidistant from the center anymore, but for that we relied on AllRADecoder.

A special thanks must be extended to artist Massimo Zonari, who lent his equipment (wheeled scaffolding) for installing stuff at an height. My own ladder would have wholly unsufficient.

Projection Mapping

Back during the Sipario projects, 4 professional grade projection screens were bought. They're still in Faenza. During this project, movers were hired to bring them to Tresigallo, but they didn't manage to get them out of the building.

Furthermore, those screens were designed to be anchored to the walls and to the ceiling, which was not possible in this building. I also wished to avoid having the metal structure between the viewer and the screen, which is something the PI actually thinks desirable.

My solution was recycling the 4 projectors, buy some cheap photography screens (they were like €30 each), and hang them in front of each side of the octagon using steel cable. We kept the "back" side open for easier access, and used the slack to make it easier to properly tension the other sides.

I wrote a simple OpenCV Python script for applying the projection. It takes 360-degree equirectangular videos, applies custom perspective warping (keystone correction) so they map onto the hanging screens, and the rest of the video feed is black.

A startup script handles common setup issues (like macOS shuffling the display order of projectors upon reboot, and setting the sound card's clock source) and can be controlled remotely via a custom web interface. It instantiates 4 copies of mpv, and puts them in full screen on the relevant projector.

1. Calibration and Rendering projection_mapper.py

This script handles the offline heavy lifting: translating raw 360° video into projector-ready files.

• View Extraction: It reads the 360-degree equirectangular source video and extracts flat, perspective views corresponding to the 4 projector positions. Since each of the 4 projectors covers two adjacent walls of the octagonal room (e.g., walls 1 & 2, 3 & 4), the script splits the field of view accordingly.
  
• Calibration GUI: The script includes an interactive calibration tool using OpenCV. First, using Blender on the Mac Studio, the borders of each screen are drawn. This allows for a "theory-less" mapping, as we're projecting at the coordinates where the screen actually is, looking at screen-space coordinates. When mounting the projectors, I didn't even take measures, they're just aimed at the general direction of the target screen. Then, we take these images, and using the script we define the plane coordinates, by clicking on the corners.
  

  

  

  This creates a transform that is stored in mapping.json.
• Perspective Warping: Based on those clicks, it generates a homography matrix cv2.getPerspectiveTransform and saves it to mapping.json. It then applies cv2.warpPerspective frame-by-frame to physically distort the video so that when it hits the walls at an angle, it looks perfectly straight to the viewer.
• Export: It outputs four synchronized 1920x1080 MP4 files for each scene output_proj_1e2.mp4

2. The Playback Engine master_player.py

This spawns 4 instances of mpv, arranges them on the 4 screens, and coordinates the playback between mpv and Bidule using OSC messages. It also creates a web interface that can be used to play, pause, and rearrange pre-recorded scenes.

• Screen Layout Resilience: macOS notoriously scrambles identical displays when the Mac Studio reboots. master_player.py solves this by integrating with the displayplacer CLI and a Swift helper script get_screens.swift. It reads the persistent Hardware UUIDs of the 4 projectors, checks their X/Y coordinates in the virtual workspace, and automatically maps the 4 video outputs to the correct physical projectors.

3. Remote Control Web UI

Built directly into master_player.py is a lightweight Python HTTPServer.

• When the installation boots, it exposes a Web UI on port 8080.
• This mobile-friendly interface allows an operator to connect via phone to select scenes (Duomo, Chitarra, Moto, Campanini, Dialetto, Quadri, etc.), see the running duration, pause/resume, restart the current video, or assemble a playlist on the fly. The problem is that this requires both the computer and the phone to be on the same LAN, and currently there isn't an available one. I've tested it by connecting the computer to my phone's hotspot and it works pretty well.

4. Initialization startup.command

This bash script is designed to make booting the installation foolproof.

• It ensures that the Zotech A2B card is set to coaxial clock source. The problem is that sometimes the TCP module of the card stops responding, so the user should power cycle the board. This is not complicated, but it is beyond what be expected of a layman.
• It forces macOS to apply a hardcoded display layout topology using a massive displayplacer string. This guarantees the virtual display arrangement is perfectly reset before the Python scripts attempt to launch the fullscreen mpv windows.
• Then, it launches Bidule, and the previously-mensioned master_player script, which in turns launches mpv.

5. Spatial Audio Pipeline (Ambisonics)

The individual speakers are RCF M05 Compact. . Complementing the Ambisonics systems, there are also the 20 soundbars constructed during the Sipario project. Their construction enthused the team so much that they started a company, called Zotech, who also built the 32 channel amplifiers. The whole system connects a single RME MADIface to a chain of these little boxes, to the soundbars, and from there to a rack-mounted 32-channel amplier.

After the Sipario project concluded, there was a flood in Faenza, and some of the equipment was lost. This included one of those MADI boxes, and the previous amplifier. Currently, there is another Zotech box in its place, but as it was designed for a different application, it defaults to an internal source for its clock. A script forces it to switch to a coaxial source, but it has to be run at every system startup.

The Ambisonics decoding is done with ALLRADecoder inside of Plogue Bidule.

A standalone layout is available, but at full regime, the room should also use our soundbars.

• 3rd Order Ambisonics (AmbiX): The master audio tracks for the scenes (e.g., chitarra_ambix_3o.wav, dialetto_ambix.wav) are produced in up to 3rd Order Ambisonics. This is a 16-channel audio format capable of representing a full 360-degree, spherical sound field with high spatial resolution, allowing the audience to precisely localize sounds corresponding to the projected visuals.
• OSC Synchronization: To keep te audio locked with the 4 video streams, master_player.py acts as the conductor. It sends Open Sound Control (OSC) messages,like /File_Player_chitarra/Play, over UDP 127.0.0.1:8000 to Bidule. When a scene is changed, paused, or restarted from the Web UI, the Python script fires OSC packets to Bidule to seek the audiofile and toggle playback seamlessly.

Right now, we're using the soundbars as 8 "wide" sources in the decoder. I should actually address them as Wave Field Synthesis sources. We're using Campanini's WFSMixer VST plugin, which requires a JSON file containing the coordinates of each speaker in the array. The actual physical arrangement of the speakers is a bit different than what was originally designed, in order to accomodate the screens. I have made a 3D scan of their positions, but I haven't extract the coordinates yet. Once that is done, we can do some pretty interesting demos, putting virtual sound sources inside the listening space, and playing around with proximity effect. This should be done by late November 2026, when we'll have the final demo.

Utility Scripts

• generate_equirectangular_mask.py Used to generate an equirectangular image that maps image portions to the screens. I'm counting the physical screens from left to right, when entering the room, so that 3 is the "front". When importing this image in a video editor, it becomes feasible to understand where the image is going to end up.



• fix_colleague_video.py: Applies spherical correction to equirectangular videos, assuming their content does not line up with the edges of the screen. It slices up the area corresponding to each screen, and gives it a geometric projection. Without this, a rectangle added to the video will simply be stretched over the virtual sphere, and its edges will be curved.
• osc_monitor.py / test_osc.py: Debugging scripts used to listen to or send raw OSC packets while setting up the Plogue Bidule integration.

Adriano Farina, 2026-09-29