oomer/vmax2bella
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
vmax2bella/README.md

10 KiB
Raw Blame History

vmax2bella

[WORK IN PROGRESS, file format is currently not fully grokked]

Command line convertor from VoxelMax .vmax to DiffuseLogic .bsz

example

Build

MacOS

mkdir workdir
git clone https://github.com/lzfse/lzfse
mkdir -p lzfse/build
cd lzfse/build
/Applications/CMake.app/Contents/bin/cmake -DCMAKE_OSX_ARCHITECTURES="x86_64;arm64" ..
make -j4
cd ../..
mkdir homebrew
curl -L https://github.com/Homebrew/brew/tarball/master | tar xz --strip-components 1 -C homebrew
eval "$(homebrew/bin/brew shellenv)"
brew update --force --quiet
brew install libtool autoconf automake
git clone https://github.com/libimobiledevice/libplist
cd libplist

./autogen.sh --prefix=$PWD/install --without-cython
make -j4
cd src/.libs
install_name_tool -id @rpath/libplist-2.0.4.dylib src/.libs/libplist-2.0.4.dylib
cd ..
git clone https://github.com/oomer/vmax2bella.git
cd vmax2bella
make

Linux

mkdir workdir
git clone https://github.com/lzfse/lzfse
mkdir lzfse/build
cd lzfse/build
cmake ..
make -j4
cd ../..
git clone https://github.com/oomer/vmax2bella.git
cd vmax2bella
make

Windows

Technical Specification: VoxelMax Format

Overview

  • This document specifies a chunked voxel storage format embedded in property list (plist) files. The format provides an efficient representation of 3D voxel data through a combination of Morton-encoded spatial indexing and a sparse representation approach.

File Structure

  • Format: Property List (plist)
  • Structure: Hierarchical key-value structure with nested dictionaries and arrays
  • plist is compressed using the LZFSE, an open source reference c implementation is here
root
└── snapshots (array)
    └── Each snapshot (dictionary)
        ├── s (dictionary) - Snapshot data
        │   ├── id (dictionary) - Identifiers
        │   │   ├── c (int64) - Chunk ID
        │   │   ├── s (int64) - Session ID
        │   │   └── t (int64) - Type ID
        │   ├── lc (binary data) - Layer Color Usage
        │   ├── ds (binary data) - Voxel data stream
        │   ├── dlc (binary data) - Deselected Layer Color Usage
        │   └── st (dictionary) - Statistics/metadata
        │       ├── c (int64) - Count of voxels in the chunk
        │       ├── sc (int64) - Selected Count (number of selected voxels)
        │       ├── smin (array) - Selected Minimum coordinates [x,y,z,w]
        │       ├── smax (array) - Selected Maximum coordinates [x,y,z,w]
        │       ├── min (array) - Minimum coordinates of all voxels [x,y,z]
        │       ├── max (array) - Maximum coordinates of all voxels [x,y,z]
        │       └── e (dictionary) - Extent
        │           ├── o (array) - Origin/reference point [x,y,z]
        │           └── s (array) - Size/dimensions [width,height,depth]

Chunking System

Volume Organization

  • The total volume is divided into chunks for efficient storage and manipulation
  • Standard chunk size: 32×32×32 voxels
  • Total addressable space: 256×256×256 voxels (8×8×8 chunks)

Morton Encoding for Chunks

  • Chunk IDs are encoded using 24 bits (8 bits per dimension)
  • This allows addressing up to 256 chunks in each dimension , although only 8x8x8 are used in practice
  • The decodeMortonChunkID function extracts x, y, z coordinates from a Morton-encoded chunk ID
  • The resulting chunk coordinates are then multiplied by 32 to get the world position of the chunk

Chunk Addressing

  • Each chunk has 3D coordinates (chunk_x, chunk_y, chunk_z)
  • For a 128×128×128 volume with 32×32×32 chunks, there would be 4×4×4 chunks total (64 chunks)
  • Chunks are only stored if they contain at least one non-empty voxel
  • Each snapshot contains data for a specific chunk, identified by the 'c' value in the 's.id' dictionary

Data Fields

Voxel Data Stream (ds)

  • Variable-length binary data

  • Contains pairs of bytes for each voxel: [position_byte, color_byte]

  • Each chunk can contain up to 32,768 voxels (32×32×32)

  • Position Byte:

    • The format uses a encoding approach that combines sequential and Morton encoding for optimal storage efficiency:
      • Uses a mix of position=0 bytes (for sequential implicit positions) and Morton-encoded position bytes
      • The decoder maintains an internal position counter that advances through the chunk in a predefined order (x, then y, then z)
      • Color byte 0 indicates "no voxel at this position" (empty space)
      • If a chunk uses the entire 256x256x256 addressable space, then it uses exactly 65,536 bytes (32,768 voxel pairs)
      • This is the dense case and it not memory efficient
      • When we introduce morton encoding we can jump to a specific position in the chunk
      • Data stream can terminate at any point, avoiding the need to store all 32,768 voxel pairs

    Morton Encoding Process

    • A space-filling curve that interleaves the bits of the x, y, and z coordinates
    • Used to convert 3D coordinates to a 1D index and vice versa
    • Creates a coherent ordering of voxels that preserves spatial locality
    1. Take the binary representation of x, y, and z coordinates
    2. Interleave the bits in the order: z₀, y₀, x₀, z₁, y₁, x₁, z₂, y₂, x₂, ...
    3. The resulting binary number is the Morton code

  • Color Byte:

    • Stores the color value + 1 (offset of +1 from actual color)
    • Value 0 indicates no voxel at this position

  • A fully populated chunk will have 32,768 voxel pairs (65,536 bytes total in ds)

Snapshot Accumulation

  • Each snapshot contains data for a specific chunk (identified by the chunk ID)
  • Multiple snapshots together build up the complete voxel model
  • Later snapshots for the same chunk ID overwrite earlier ones, allowing for edits over time

Layer Color Usagw (lc)

  • s.lc is a summary table (256 bytes) that tracks which colors are used anywhere in the chunk
  • Each byte position (0-255) corresponds to a color palette ID
  • [TODO] understand why the word layer color is used, what is a layer color

Deselected Layer Color Usage (dlc)

  • Optional 256-byte array
  • Used during editing to track which color layers the user has deselected
  • Primarily for UI state preservation rather than 3D model representation

Statistics Data (st)

  • Dictionary containing metadata about the voxels in a chunk:
    • c (count): Total number of voxels in the chunk
    • sc (selectedCount): Number of currently selected voxels
    • sMin (selectedMin): Array defining minimum coordinates of current selection [x,y,z,w]
    • sMax (selectedMax): Array defining maximum coordinates of current selection [x,y,z,w]
    • min: Array defining minimum coordinates of all voxels [x,y,z]
    • max: Array defining maximum coordinates of all voxels [x,y,z]
    • e (extent): Array defining the bounding box [min_x, min_y, min_z, max_x, max_y, max_z]
    • e.o (extent.origin): Reference point or offset for extent calculations

Coordinate Systems

Primary Coordinate System

  • Y-up coordinate system: Y is the vertical axis
  • Origin (0,0,0) is at the bottom-left-front corner
  • Coordinates increase toward right (X+), up (Y+), and backward (Z+)

Addressing Scheme

  1. World Space: Absolute coordinates in the full volume
  2. Chunk Space: Which chunk contains a voxel (chunk_x, chunk_y, chunk_z)
  3. Local Space: Coordinates within a chunk (local_x, local_y, local_z)

Coordinate Conversion

  • World to Chunk:
    • chunk_x = floor(world_x / 32)
    • chunk_y = floor(world_y / 32)
    • chunk_z = floor(world_z / 32)
  • World to Local:
    • local_x = world_x % 32
    • local_y = world_y % 32
    • local_z = world_z % 32
  • Chunk+Local to World:
    • world_x = chunk_x * 32 + local_x
    • world_y = chunk_y * 32 + local_y
    • world_z = chunk_z * 32 + local_z

Detailed Morton Encoding Example

To clarify the bit interleaving process, here's a step-by-step example:

For position (3,1,2):

  1. Convert to binary (3 bits each):

    • x = 3 = 011 (bits labeled as x₂x₁x₀)
    • y = 1 = 001 (bits labeled as y₂y₁y₀)
    • z = 2 = 010 (bits labeled as z₂z₁z₀)

  2. Interleave the bits in the order z₂y₂x₂, z₁y₁x₁, z₀y₀x₀:

    • z₂y₂x₂ = 001 (z₂=0, y₂=0, x₂=1)
    • z₁y₁x₁ = 010 (z₁=1, y₁=0, x₁=0)
    • z₀y₀x₀ = 100 (z₀=0, y₀=0, x₀=0)

  3. Combine: 001010100 = binary 10000110 = decimal 134

Therefore, position (3,1,2) has Morton index 134.

Implementation Guidance

Reading Algorithm

  1. Parse the plist file to access the snapshot array
  2. For each snapshot: a. Extract the chunk ID from s > id > c b. Extract the lc and ds data c. Process the ds data in pairs of bytes (position, color) d. Calculate the world origin by decoding the Morton chunk ID and multiplying by 32 e. Store the voxels for this chunk ID
  3. Combine all snapshots to build the complete voxel model, using the chunk IDs as keys

Writing Algorithm

  1. Organize voxels by chunk (32×32×32 voxels per chunk)
  2. For each non-empty chunk: a. Create a snapshot entry b. Set up the id dictionary with the appropriate chunk ID c. Set up a 256-byte lc array (all zeros) d. Create the ds data by encoding each voxel as a (position, color+1) pair e. Set the appropriate byte in lc to 1 if the color is used in ds
  3. Add all snapshots to the array
  4. Write the complete structure to a plist file
  • [?] Models typically use SessionIDs to group related edits (observed values include 10 and 18)

Snapshot Types

The 't' field in the snapshot's 's.id' dictionary indicates the type of snapshot:

  • 0: underRestore - Snapshot being restored from a previous state
  • 1: redoRestore - Snapshot being restored during a redo operation
  • 2: undo - Snapshot created for an undo operation
  • 3: redo - Snapshot created for a redo operation
  • 4: checkpoint - Snapshot created as a regular checkpoint during editing (most common)
  • 5: selection - Snapshot representing a selection operation