Posted on

I Used 4DV AI to Reimagine Dynamic Scenes — Here’s What I Learned

4DV AI

When I first heard about 4DV AI and its 4D Gaussian Splatting capabilities, I knew I had to try it. As someone constantly building immersive experiences in XR, the idea of rendering real-time dynamic scenes without the usual NeRF slowdowns was too tempting to pass up.

Let me walk you through what it was like using 4DV AI—and why I think it’s a game-changer for anyone working in AR/VR, volumetric video, or digital twins.

Capturing Time: Not Just Space

One of the biggest limitations with tools like NeRF is their struggle with motion. Scenes look stunning—but only if everything stands still. 4DV AI breaks that barrier.

Unlike traditional 3D capture, this tool doesn’t just reconstruct static geometry. It captures how scenes move through time, transforming video into 4D Gaussian splats that update in real time.

I tested it using a short clip of a colleague walking through a worksite. Within minutes, 4DV AI generated a fluid, motion-consistent reconstruction. No stutters. No weird motion artifacts. Just smooth, time-aware rendering.

It felt like watching a volumetric video made of light particles.

💡 Want to learn about related advances in photogrammetry? Check out RealityScan 2.0 — another AI-driven leap in 3D capture.

The Setup: Surprisingly Simple

What surprised me most was how accessible the workflow was. All I needed was:

  • A multi-view camera setup (even basic GoPros work)

  • Calibrated timestamps for synchronization

  • The 4DV pipeline and a GPU that doesn’t melt

Once you load the data, the system creates temporal Gaussian splats—each one carrying info about shape, color, and how it moves over time. It’s like training a neural memory of how the scene evolves, rather than capturing frozen moments.

The optimization even includes temporal consistency loss, so objects maintain shape and coherence as they move across frames. I was reminded of similar pipeline design I used in VR maintenance training, but this felt more fluid and organic.

What I Saw: A Living Scene

There’s something surreal about moving a virtual camera through a 4D splat space. Unlike mesh-based reconstructions that feel rigid or synthetic, this felt… natural. Like I was looking through a lens into a memory.

Whether it was subtle gestures, flowing clothes, or soft shadow transitions—everything stayed consistent, without temporal flickering or weird blending. And it all ran in real-time, thanks to the rasterization-based splatting pipeline.

If you’ve experimented with tools like Midjourney for 3D generation, imagine that—but live, interactive, and constantly evolving.

How This Will Change XR (and How I’ll Use It)

In my work with BIM AR visualizations, we often struggle to make dynamic environments feel believable. With 4DV AI, I can finally integrate human motion, machinery, and environmental changes—without building entire simulation rigs.

Here’s where I see this going:

  • Event captures: Create holographic replays of real events

  • VR storytelling: Time-travel scenes where environments evolve

  • Training: Real-world movements rendered into XR training modules

  • Remote presence: Volumetric avatars that don’t break immersion

If you’re working in VR fire safety or industrial simulation, this is a tool to keep an eye on.

Final Thoughts: More Than Just a Flex

Trying out 4DV AI wasn’t just fun—it showed me what’s next. Scene representation isn’t about static geometry anymore. It’s about memory. Time. Realism that evolves.

And if you’re building the future of XR like I am, you’ll want your virtual worlds to breathe.

Posted on

Meta Spatial SDK Upgrade: A New Era of Mixed Reality Development

Meta Spatial SDK Upgrade

Meta has rolled out a major upgrade to its Spatial SDK, and it’s one of the most impactful updates in the XR space this year. If you’re building for the Quest 3 or preparing for what’s next in mixed reality, this upgrade is your launchpad.

With improvements to spatial anchoring, shared spaces, and hand tracking, and new features like Passthrough Camera Access and the Horizon OS UI Set, this update is more than just a patch — it’s a foundation for the future of immersive computing.

Explore XR architectures: Architecture of Virtual Reality System

What’s New in the Meta Spatial SDK?

1. Smarter Spatial Anchors

The updated SDK boosts spatial anchor accuracy and persistence, even across sessions or devices. This means your AR content can now stay fixed in the real world — great for room-scale games or industrial training.

→ Learn more: Meta Spatial Anchors Guide

🧑‍🤝‍🧑 2. Shared Spaces for Multi-User Experiences

Multiple users can now interact in a synchronized, shared physical space using Shared Spatial Anchors. This is a huge leap for multiplayer XR games and collaborative training.

→ Related post: Shared Spaces: Multiplayer XR on Web

3. Hand Tracking 2.2 – Now Even More Natural

AI-driven updates now make gesture recognition faster and more reliable, reducing latency and boosting precision — especially useful for controller-free applications.

→ Insightful read: Top VR Haptic Gloves in 2025

→ Details: Meta Hand Tracking 2.2 Update

4. Passthrough Camera Access + AI

Developers can now integrate real-world visuals directly into apps, enabling AR-style interactions in MR environments. The Meta Spatial Scanner showcase even combines this with AI object recognition using LLAMA 3.2.

→ Example: Reddit discussion on new Spatial SDK

5. Horizon OS UI Set

Meta now provides a full UI toolkit for developers — with customizable buttons, dropdowns, and more — helping you speed up interface design without reinventing the wheel.

→ Dev Guide: Horizon UI Overview

6. Import Support for 3D Models

While there’s no direct RealityScan 2.0 integration, the SDK does support popular formats like .glb, .fbx, and .obj, allowing you to use scanned 3D assets easily.

→ Related read: RealityScan 2.0 – AI 3D Reconstruction Tool

Why This Upgrade Changes the Game

Whether you’re developing VR Maintenance Training or creating branded experiences at events, the Spatial SDK is designed to give you:

  • Greater stability for persistent AR overlays
  • Smarter interactions through AI + passthrough
  • Rapid prototyping with pre-built UI elements
  • Multi-user synchronicity for collaborative apps

With these features, you’re not just building apps — you’re building experiences that feel native to the real world.

Why This Upgrade Changes the Game

Meta offers full documentation, code samples, and Unity integration guides. Developers can also experiment with shared experiences using platforms like Shared Spaces on GitHub.

If you’re new to VR development, check out our curated list of Beginner to Advanced VR Courses to get started.

Final Thoughts

The Meta Spatial SDK upgrade isn’t just an evolution — it’s a shift in how we design for space, context, and connection. As hardware catches up with software intelligence, tools like this will define what’s next in XR.

Ready to build? The future of spatial computing is already here. You just need the right SDK.

Posted on

Exploring Shared Spaces: Open-Source Multiplayer XR for the Web

Introduction: The Rise of Collaborative XR on the Web

In the evolving world of extended reality (XR), the ability to collaborate in real-time within shared virtual or augmented environments is becoming essential—whether for remote work, design reviews, education, or interactive exhibitions. While platforms like Spatial and Mozilla Hubs offer hosted solutions, developers looking for a lightweight, self-hosted, and customizable alternative often run into complex architecture challenges.

That’s where Shared Spaces comes in—an open-source project that makes building WebXR multiplayer environments incredibly accessible. It provides a simple yet powerful framework that uses WebSockets, WebXR, and Three.js to sync headsets, hand tracking, and avatars between users in real-time, all directly in the browser.

What is Shared Spaces?

Shared Spaces is a WebXR-compatible, open-source multiplayer platform designed for collaborative XR environments. Created by Christophe Cabanier, a long-time contributor to web-based XR initiatives, this project allows multiple users to join a shared room and see each other’s position, orientation, and hand gestures in real time—all without installing anything beyond a browser.

Key features:

    • Browser-based (no installs)
    • WebXR support (VR headsets like Quest work out of the box)
    • WebSocket-based server for low-latency data transfer
    • Lightweight avatar and controller tracking
    • Three.js for rendering 3D scenes

Why Shared Spaces Matters?

Shared Spaces fills a crucial gap in the ecosystem: lightweight, self-hosted WebXR multiplayer. Most WebXR demos focus on single-user experiences. Shared Spaces gives developers a practical foundation to explore use cases like:

  • Virtual exhibitions
  • Collaborative design sessions
  • Remote team meetings
  • Educational or museum AR/VR setups

You can host your own server or modify the codebase for deeper integrations—ideal for companies looking to build branded XR apps without relying on third-party platforms.

How It Works

Shared Spaces uses:

  • WebXR for accessing XR hardware (e.g., Quest, Vive, browser-based AR).
  • Three.js for rendering immersive 3D environments.
  • WebSockets to send real-time updates (headset pose, hand position, etc.) to all connected clients.
  • Node.js + Express to run the backend WebSocket server.

Each client tracks their own pose and sends it to the server, which rebroadcasts it to all others in the room. This keeps the system lightweight, fast, and suitable even for mobile devices or standalone headsets.

Developer Experience

For developers, getting started is surprisingly simple:

  1. Clone the repo: git clone https://github.com/cabanier/shared-spaces.git
  2. Install dependencies: npm install
  3. Start the server: node server.js
  4. Open the app in your WebXR-supported browser

No Unity, Unreal, or heavy backend—just clean JavaScript and real-time 3D.

Use Cases and Applications

Shared Spaces can be the foundation for:

  • Bespoke branded virtual rooms
  • Training simulations
  • Augmented reality tours using WebAR
  • Virtual showrooms for products or real estate
  • Art installations or collaborative storytelling

For XR agencies and indie developers alike, this is a practical project to explore or extend.

Conclusion: Lightweight Multiplayer XR is Here

Shared Spaces is a great example of how powerful web-based XR has become. By combining the simplicity of Three.js with the power of WebXR and WebSockets, it allows any developer to create immersive, shared experiences in minutes.

If you’re exploring XR for collaboration, prototyping, or brand engagement—Shared Spaces offers a perfect starting point that’s both developer-friendly and future-ready.

Further Reading & Resources

  1. GitHub: Shared Spaces Repository
  2. WebXR Device API
  3. Three.js Documentation
  4. WebSocket Protocol
Posted on

RealityScan 2.0 Unveiled: AI-Powered 3D Reconstruction Tool by Epic Games

Reality Scan

RealityScan 2.0 emerges as a game-changer for professionals in architecture, visual effects, and game development. By integrating advanced features like GPU-accelerated alignment and AI-driven masking, it addresses longstanding challenges in 3D reconstruction workflows.

Cutting-Edge Features Enhancing Workflow

1. Enhanced Alignment System
The new default high-quality feature detection mode ensures tighter camera alignments with fewer disjointed components. This improvement is crucial for workflows involving Gaussian Splatting, reducing ghosting artifacts and minimizing the need for extensive post-processing.

2. AI-Based Masking
RealityScan 2.0 introduces an AI-powered segmentation model that efficiently separates subjects from backgrounds. This feature eliminates the tedious process of manual masking, allowing for seamless iterations within the application.

3. Quality Analysis Mode
A new analysis mode provides intuitive color overlays on sparse point clouds and meshes, indicating areas of strong (green) and weak (red) coverage. This visualization aids in identifying and addressing gaps before final rendering or splat conversion.

4. Aerial LiDAR Support
The update adds native support for aerial LiDAR files, including formats like LAS, LAZ, and E57. This capability enables the integration of unordered point clouds from drones directly into the reconstruction pipeline, facilitating hybrid workflows that combine color information from Gaussian Splatting with LiDAR accuracy.

Streamlined, High-Quality 3D Reconstructions

With these enhancements, RealityScan 2.0 positions itself as an indispensable tool for professionals seeking efficient and high-fidelity 3D reconstructions. The combination of GPU acceleration, AI integration, and comprehensive analysis tools streamlines the workflow, saving time and improving output quality.

Integrate RealityScan 2.0 into Your Workflow

RealityScan 2.0 is expected to be available through the Epic Games Launcher by the end of June 2025. Currently, RealityScan remains free for individuals and companies below a certain revenue threshold. To explore the current version and prepare for the upcoming release, visit the Epic Games Launcher.

Posted on

First Interview in the Metaverse: Zuck’s Photorealistic Codec Avatars

First Interview in the Metaverse of Lex Fridman Podcast

While critics have spent the previous four years busy writing obituaries for Meta’s metaverse dream, Mark Zuckerberg’s most recent demonstration of its photorealistic avatars suggests it might not be quite as dead as thought. While discussing photorealistic avatars, it’s essential to highlight that Twin Reality’s Virtual Reality Industrial Training empowers businesses and organizations to provide their employees with highly realistic and immersive training environments, facilitating the attainment of photorealistic VR experiences.

On a Sept. 28 episode of the Lex Fridman podcast, Zuckerberg and the popular computer scientist had a one-hour face-to-face talk. Only, it wasn’t actually in person at all.

Instead, the whole conversation between Fridman and Zuckerberg took place in the metaverse using lifelike avatars, made possible by Meta’s Quest 3 headsets and noise-cancelling headphones.

The technology on display is the newest version of Codec Avatars. One of Meta’s longest-running research initiatives, Codec Avatars was first unveiled in 2019. Its goal is to develop totally photorealistic, real-time avatars that operate through headsets with facial tracking sensors.

However, according to Zuckerberg, customers could have to wait a few years before donning their own lifelike avatars. He explained that the technology involved entails pricey machine learning software and thorough head scans by specialised equipment with more than 100 distinct cameras.

He estimated that it would take at least three years before this was accessible to everyday consumers.

However, Zuckerberg emphasised that the business intends to remove as many obstacles as possible, adding that in the future, these scans might be achievable with a regular smartphone.