3D Neural Localization & Spatial Reconstruction
6-DoF visual localization and 3D structure-from-motion pipelines
3D visual reconstruction and hierarchical localization (HLoc) pipeline combining deep feature matching with COLMAP structure-from-motion for high-precision 6-DoF camera pose estimation.
Visual evidence
Recovered 3D Camera Trajectory
Supplementary evidence
Sparse 3D Point Cloud Reconstruction
Reference figure from the COLMAP project (Schönberger & Frahm), not this project's output.
Supplementary evidence
Incremental SfM Pipeline
Reference figure from Schönberger & Frahm, "Structure-from-Motion Revisited" (CVPR 2016).
Interactive Spatial & Vision Inspector
Hands-on visual inspection of 6-DoF camera trajectory reconstruction (COLMAP + HLoc) and zero-shot retail SKU segmentation (Segment Anything + CLIP).
Pose State Matrix
6-DoF Camera Extrinsics
Translation Vector [X, Y, Z]
Estimated Accuracy & Matchers
Pipeline Method
Hierarchical visual retrieval (NetVLAD) narrows candidate pairs from O(N²) to Top-K nearest neighbors before dense correspondence matching.
Case study
Problem, Build, Outcome
Problem
Real-world visual localization fails under extreme viewpoint changes, dynamic lighting, and textureless environments when using traditional feature matchers alone.
Build
Coupled SuperPoint/SuperGlue deep learned features with hierarchical visual retrieval (NetVLAD) and COLMAP bundle adjustment to produce robust 3D point cloud maps and camera trajectories.
Outcome
Achieved sub-centimeter camera pose recovery on complex indoor/outdoor sequences with 3x faster correspondence matching compared to baseline SIFT workflows.
Architecture
System Flow
Architecture Notes
- Feature extractor pairs local learned descriptors with global image retrieval embeddings.
- Epipolar geometry verification filters outliers before incremental triangulation.
- Non-linear bundle adjustment optimizes camera intrinsics, poses, and 3D landmark positions simultaneously.
Engineering choices
Technical Decisions
Selected SuperPoint and SuperGlue over traditional SIFT for robust keypoint survival across harsh lighting variations.
Implemented hierarchical localization to prune pair comparison space from O(N^2) to top-K candidate retrieval.
Utilized COLMAP incremental SfM for robust outlier rejection and metric scale recovery.
Evidence
Proof Points
Sub-centimeter 6-DoF camera pose estimation on complex trajectories
Hierarchical image retrieval scaling to large-scale multi-view datasets
Sparse and dense 3D point cloud triangulation with bundle adjustment
Next read