244 lines
9.8 KiB
Markdown
244 lines
9.8 KiB
Markdown
# vmax2bella
|
||
[WORK IN PROGRESS, file format is currently not fully grokked]
|
||
|
||
Command line convertor from VoxelMax .vmax to DiffuseLogic .bsz
|
||
|
||
|
||
|
||
|
||
|
||
# Build
|
||
|
||
# MacOS
|
||
|
||
```
|
||
mkdir workdir
|
||
git clone https://github.com/lzfse/lzfse
|
||
mkdir -p lzfse/build
|
||
cd lzfse/build
|
||
/Applications/CMake.app/Contents/bin/cmake ..
|
||
make -j4
|
||
cd ../..
|
||
mkdir homebrew
|
||
curl -L https://github.com/Homebrew/brew/tarball/master | tar xz --strip-components 1 -C homebrew
|
||
eval "$(homebrew/bin/brew shellenv)"
|
||
brew update --force --quiet
|
||
brew install libtool autoconf automake
|
||
git clone https://github.com/libimobiledevice/libplist
|
||
cd libplist
|
||
./autogen.sh
|
||
make -j4
|
||
cd ..
|
||
git clone https://github.com/oomer/vmax2bella.git
|
||
cd vmax2bella
|
||
make
|
||
```
|
||
|
||
# Linux
|
||
|
||
```
|
||
mkdir workdir
|
||
git clone https://github.com/lzfse/lzfse
|
||
mkdir lzfse/build
|
||
cd lzfse/build
|
||
cmake ..
|
||
make -j4
|
||
cd ../..
|
||
git clone https://github.com/oomer/vmax2bella.git
|
||
cd vmax2bella
|
||
make
|
||
```
|
||
|
||
# Windows
|
||
```
|
||
|
||
|
||
```
|
||
|
||
---
|
||
|
||
# Technical Specification: VoxelMax Format
|
||
|
||
## Overview
|
||
- This document specifies a chunked voxel storage format embedded in property list (plist) files. The format provides an efficient representation of 3D voxel data through a combination of Morton-encoded spatial indexing and a sparse representation approach.
|
||
|
||
## File Structure
|
||
- Format: Property List (plist)
|
||
- Structure: Hierarchical key-value structure with nested dictionaries and arrays
|
||
- plist is compressed using the LZFSE, an open source reference c implementation is [here](https://github.com/lzfse/lzfse)
|
||
|
||
```
|
||
root
|
||
└── snapshots (array)
|
||
└── Each snapshot (dictionary)
|
||
├── s (dictionary) - Snapshot data
|
||
│ ├── id (dictionary) - Identifiers
|
||
│ │ ├── c (int64) - Chunk ID
|
||
│ │ ├── s (int64) - Session ID
|
||
│ │ └── t (int64) - Type ID
|
||
│ ├── lc (binary data) - Layer Color Usage
|
||
│ ├── ds (binary data) - Voxel data stream
|
||
│ ├── dlc (binary data) - Deselected Layer Color Usage
|
||
│ └── st (dictionary) - Statistics/metadata
|
||
│ ├── c (int64) - Count of voxels in the chunk
|
||
│ ├── sc (int64) - Selected Count (number of selected voxels)
|
||
│ ├── smin (array) - Selected Minimum coordinates [x,y,z,w]
|
||
│ ├── smax (array) - Selected Maximum coordinates [x,y,z,w]
|
||
│ ├── min (array) - Minimum coordinates of all voxels [x,y,z]
|
||
│ ├── max (array) - Maximum coordinates of all voxels [x,y,z]
|
||
│ └── e (dictionary) - Extent
|
||
│ ├── o (array) - Origin/reference point [x,y,z]
|
||
│ └── s (array) - Size/dimensions [width,height,depth]
|
||
```
|
||
|
||
## Chunking System
|
||
### Volume Organization
|
||
- The total volume is divided into chunks for efficient storage and manipulation
|
||
- Standard chunk size: 32×32×32 voxels
|
||
- Total addressable space: 256×256×256 voxels (8×8×8 chunks)
|
||
|
||
### Morton Encoding for Chunks
|
||
- Chunk IDs are encoded using 24 bits (8 bits per dimension)
|
||
- This allows addressing up to 256 chunks in each dimension , although only 8x8x8 are used in practice
|
||
- The decodeMortonChunkID function extracts x, y, z coordinates from a Morton-encoded chunk ID
|
||
- The resulting chunk coordinates are then multiplied by 32 to get the world position of the chunk
|
||
|
||
### Chunk Addressing
|
||
- Each chunk has 3D coordinates (chunk_x, chunk_y, chunk_z)
|
||
- For a 128×128×128 volume with 32×32×32 chunks, there would be 4×4×4 chunks total (64 chunks)
|
||
- Chunks are only stored if they contain at least one non-empty voxel
|
||
- Each snapshot contains data for a specific chunk, identified by the 'c' value in the 's.id' dictionary
|
||
|
||
## Data Fields
|
||
### Voxel Data Stream (ds)
|
||
- Variable-length binary data
|
||
- Contains pairs of bytes for each voxel: [position_byte, color_byte]
|
||
- Each chunk can contain up to 32,768 voxels (32×32×32)
|
||
- *Position Byte:*
|
||
- The format uses a encoding approach that combines sequential and Morton encoding for optimal storage efficiency:
|
||
- Uses a mix of position=0 bytes (for sequential implicit positions) and Morton-encoded position bytes
|
||
- The decoder maintains an internal position counter that advances through the chunk in a predefined order (x, then y, then z)
|
||
- Color byte 0 indicates "no voxel at this position" (empty space)
|
||
- If a chunk uses the entire 256x256x256 addressable space, then it uses exactly 65,536 bytes (32,768 voxel pairs)
|
||
- This is the dense case and it not memory efficient
|
||
- When we introduce morton encoding we can jump to a specific position in the chunk
|
||
- Data stream can terminate at any point, avoiding the need to store all 32,768 voxel pairs
|
||
### Morton Encoding Process
|
||
- A space-filling curve that interleaves the bits of the x, y, and z coordinates
|
||
- Used to convert 3D coordinates to a 1D index and vice versa
|
||
- Creates a coherent ordering of voxels that preserves spatial locality
|
||
1. Take the binary representation of x, y, and z coordinates
|
||
2. Interleave the bits in the order: z₀, y₀, x₀, z₁, y₁, x₁, z₂, y₂, x₂, ...
|
||
3. The resulting binary number is the Morton code
|
||
|
||
|
||
- *Color Byte:*
|
||
- Stores the color value + 1 (offset of +1 from actual color)
|
||
- Value 0 indicates no voxel at this position
|
||
- A fully populated chunk will have 32,768 voxel pairs (65,536 bytes total in ds)
|
||
|
||
### Snapshot Accumulation
|
||
- Each snapshot contains data for a specific chunk (identified by the chunk ID)
|
||
- Multiple snapshots together build up the complete voxel model
|
||
- Later snapshots for the same chunk ID overwrite earlier ones, allowing for edits over time
|
||
|
||
### Layer Color Usagw (lc)
|
||
- s.lc is a summary table (256 bytes) that tracks which colors are used anywhere in the chunk
|
||
- Each byte position (0-255) corresponds to a color palette ID
|
||
- [TODO] understand why the word layer color is used, what is a layer color
|
||
|
||
### Deselected Layer Color Usage (dlc)
|
||
- Optional 256-byte array
|
||
- Used during editing to track which color layers the user has deselected
|
||
- Primarily for UI state preservation rather than 3D model representation
|
||
|
||
### Statistics Data (st)
|
||
- Dictionary containing metadata about the voxels in a chunk:
|
||
- c (count): Total number of voxels in the chunk
|
||
- sc (selectedCount): Number of currently selected voxels
|
||
- sMin (selectedMin): Array defining minimum coordinates of current selection [x,y,z,w]
|
||
- sMax (selectedMax): Array defining maximum coordinates of current selection [x,y,z,w]
|
||
- min: Array defining minimum coordinates of all voxels [x,y,z]
|
||
- max: Array defining maximum coordinates of all voxels [x,y,z]
|
||
- e (extent): Array defining the bounding box [min_x, min_y, min_z, max_x, max_y, max_z]
|
||
- e.o (extent.origin): Reference point or offset for extent calculations
|
||
|
||
## Coordinate Systems
|
||
### Primary Coordinate System
|
||
- Y-up coordinate system: Y is the vertical axis
|
||
- Origin (0,0,0) is at the bottom-left-front corner
|
||
- Coordinates increase toward right (X+), up (Y+), and backward (Z+)
|
||
|
||
### Addressing Scheme
|
||
1. World Space: Absolute coordinates in the full volume
|
||
2. Chunk Space: Which chunk contains a voxel (chunk_x, chunk_y, chunk_z)
|
||
3. Local Space: Coordinates within a chunk (local_x, local_y, local_z)
|
||
|
||
## Coordinate Conversion
|
||
- *World to Chunk:*
|
||
- chunk_x = floor(world_x / 32)
|
||
- chunk_y = floor(world_y / 32)
|
||
- chunk_z = floor(world_z / 32)
|
||
- *World to Local:*
|
||
- local_x = world_x % 32
|
||
- local_y = world_y % 32
|
||
- local_z = world_z % 32
|
||
- *Chunk+Local to World:*
|
||
- world_x = chunk_x * 32 + local_x
|
||
- world_y = chunk_y * 32 + local_y
|
||
- world_z = chunk_z * 32 + local_z
|
||
|
||
|
||
### Detailed Morton Encoding Example
|
||
To clarify the bit interleaving process, here's a step-by-step example:
|
||
|
||
For position (3,1,2):
|
||
|
||
1. Convert to binary (3 bits each):
|
||
- x = 3 = 011 (bits labeled as x₂x₁x₀)
|
||
- y = 1 = 001 (bits labeled as y₂y₁y₀)
|
||
- z = 2 = 010 (bits labeled as z₂z₁z₀)
|
||
|
||
2. Interleave the bits in the order z₂y₂x₂, z₁y₁x₁, z₀y₀x₀:
|
||
- z₂y₂x₂ = 001 (z₂=0, y₂=0, x₂=1)
|
||
- z₁y₁x₁ = 010 (z₁=1, y₁=0, x₁=0)
|
||
- z₀y₀x₀ = 100 (z₀=0, y₀=0, x₀=0)
|
||
|
||
3. Combine: 001010100 = binary 10000110 = decimal 134
|
||
|
||
Therefore, position (3,1,2) has Morton index 134.
|
||
|
||
## Implementation Guidance
|
||
### Reading Algorithm
|
||
1. Parse the plist file to access the snapshot array
|
||
2. For each snapshot:
|
||
a. Extract the chunk ID from s > id > c
|
||
b. Extract the lc and ds data
|
||
c. Process the ds data in pairs of bytes (position, color)
|
||
d. Calculate the world origin by decoding the Morton chunk ID and multiplying by 32
|
||
e. Store the voxels for this chunk ID
|
||
3. Combine all snapshots to build the complete voxel model, using the chunk IDs as keys
|
||
|
||
### Writing Algorithm
|
||
1. Organize voxels by chunk (32×32×32 voxels per chunk)
|
||
2. For each non-empty chunk:
|
||
a. Create a snapshot entry
|
||
b. Set up the id dictionary with the appropriate chunk ID
|
||
c. Set up a 256-byte lc array (all zeros)
|
||
d. Create the ds data by encoding each voxel as a (position, color+1) pair
|
||
e. Set the appropriate byte in lc to 1 if the color is used in ds
|
||
3. Add all snapshots to the array
|
||
4. Write the complete structure to a plist file
|
||
|
||
- [?] Models typically use SessionIDs to group related edits (observed values include 10 and 18)
|
||
|
||
## Snapshot Types
|
||
The 't' field in the snapshot's 's.id' dictionary indicates the type of snapshot:
|
||
- 0: underRestore - Snapshot being restored from a previous state
|
||
- 1: redoRestore - Snapshot being restored during a redo operation
|
||
- 2: undo - Snapshot created for an undo operation
|
||
- 3: redo - Snapshot created for a redo operation
|
||
- 4: checkpoint - Snapshot created as a regular checkpoint during editing (most common)
|
||
- 5: selection - Snapshot representing a selection operation
|
||
|