NICE-SLAM
Related docs: NeRF-SLAM family overview · iMAP · Co-SLAM and ESLAM · Splat-SLAM · GS-SLAM and MonoGS · MASt3R-SLAM · 4DNDF · KISS-SLAM · Aggregated Map Semantic Segmentation · Feed-Forward 3D Reconstruction and Splatting · Volume Rendering, Radiance Fields, and Gaussian Splatting · Lie Groups — SE(3), SO(3), Jacobians
Last updated: 2026-05-24
What It Is
NICE-SLAM ("Neural Implicit Scalable Encoding for SLAM") is a dense RGB-D SLAM system published at CVPR 2022 by Zihan Zhu, Songyou Peng, Viktor Larsson, Weiwei Xu, Hujun Bao, Zhaopeng Cui, Martin R. Oswald, and Marc Pollefeys (Zhejiang University, ETH Zurich, Lund University, and Microsoft). arXiv:2112.12130.
The paper's central contribution is replacing iMAP's single global MLP with a four-level hierarchical feature grid decoded by small, pre-trained MLPs. This architectural shift solved iMAP's two fundamental problems — catastrophic forgetting over large scenes and blurry reconstruction — and established the hierarchical local-feature paradigm that every subsequent neural-implicit SLAM method builds upon.
Full citation: Zihan Zhu, Songyou Peng, Viktor Larsson, Weiwei Xu, Hujun Bao, Zhaopeng Cui, Martin R. Oswald, Marc Pollefeys. "NICE-SLAM: Neural Implicit Scalable Encoding for SLAM." IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022, pp. 12786–12796. arXiv:2112.12130.
Code: https://github.com/cvg/nice-slam (Apache-2.0; includes iMAP* re-implementation for direct comparison).
Project page: https://pengsongyou.github.io/nice-slam
Historical Context
The directly preceding work, iMAP (Sucar et al., ICCV 2021, arXiv:2103.12352), proved that a single small MLP trained live could serve as the sole scene representation in a SLAM loop. iMAP demonstrated real-time tracking and dense reconstruction from an RGB-D stream — a proof of concept for the entire neural-implicit-SLAM direction. However it had two deep architectural limits:
- Capacity. One tiny MLP encoding the entire scene cannot capture fine geometric detail at room scale. Results were blurry and over-smoothed.
- Scalability. A single global MLP must be updated globally whenever new observations arrive. New observations in one room catastrophically forget geometry in another. iMAP failed on multi-room scenes.
NICE-SLAM's thesis: replacing the global MLP with hierarchical multi-resolution feature grids decoded by small, pre-trained MLPs eliminates both problems. Local updates to one region of the grid do not propagate to unrelated regions. Pre-trained decoder weights (from ConvONet training on synthetic indoor meshes) provide geometric inductive biases that accelerate per-frame convergence. The result was the first neural-implicit SLAM system that scaled beyond single small rooms and produced reconstruction quality surpassing classical TSDF fusion.
NICE-SLAM appeared in December 2021 (arXiv) and June 2022 (CVPR). It immediately became the standard baseline against which all subsequent neural-implicit SLAM papers measured themselves.
Core Technical Idea
iMAP encodes the entire scene in one MLP: (x, y, z) -> (sigma, color). Any new observation requires retraining that single network — even a gradient step for region A corrupts the representation of region B.
NICE-SLAM's key insight: decompose scene representation into a hierarchy of spatial feature grids, each decoded by an independent small MLP. When a new frame observes region A, only the grid cells overlapping A receive gradient updates. Region B's cells are untouched. This spatial locality eliminates the catastrophic-forgetting bottleneck without any change to the optimization algorithm.
The hierarchical structure adds a second advantage: different spatial frequencies are handled at different scales. The coarse grid captures global room shape; the fine grid captures surface detail. This coarse-to-fine structure also enables tracking recovery after frame loss — the coarse level renders a plausible prediction even when the fine level is uninitialized.
The name unpacks as: Neural Implicit sCalable Encoding for SLAM.
Architecture
Four-Level Hierarchical Feature Grids
NICE-SLAM maintains four dense voxel feature grids covering the scene at different spatial resolutions:
| Level | Role | Voxel side (Replica/ScanNet) | Voxel side (TUM) |
|---|---|---|---|
| Coarse (l=0) | Coarse occupancy / shape prior | 2 m | 2 m |
| Mid (l=1) | Occupancy refinement | 32 cm | 16 cm |
| Fine (l=2) | Geometric detail | 16 cm | 8 cm |
| Color (g) | Appearance / RGB | 16 cm (same as fine) | 8 cm |
Each voxel corner stores a 32-dimensional feature vector. For an arbitrary query point p, the feature at each level is obtained by trilinear interpolation from the 8 surrounding voxel corners.
Total grid memory for a single Replica room: 12.02 MB across all four levels — far more compact than naive dense voxel maps while capturing full room geometry.
Decoder MLPs
Four independent shallow MLPs decode interpolated grid features into scalar occupancy and RGB color:
f^0— coarse occupancy decoder (pre-trained from ConvONet)f^1— mid occupancy decoder (pre-trained from ConvONet)f^2— fine occupancy decoder (fine-level features concatenated with mid-level; pre-trained)g_omega— color decoder (learned from scratch online during SLAM)
All four decoders use 5 fully-connected blocks with 32 hidden units. The coarse decoder uses no positional encoding. The mid and fine decoders use learnable Gaussian positional encoding to capture higher-frequency geometry. The fine decoder applies a zero-residual constraint: when fine-level features are zeroed (unobserved), the fine decoder outputs zero residual over the mid-level prediction, ensuring gradual refinement without spurious hallucinations.
Pre-training: Coarse and mid decoders are pre-trained jointly with a point-cloud encoder (ConvONet framework) on a synthetic indoor scene dataset derived from ShapeNet room meshes, using binary cross-entropy loss. After pre-training, only the decoder MLP weights are retained; the scene-specific grid features are what SLAM optimizes online.
Feature Lookup and Concatenation
For a query point p:
Geometry path (fine occupancy):
feat_mid(p) = trilinear_interp(grid_mid, p) -- 32-dim
feat_fine(p) = trilinear_interp(grid_fine, p) -- 32-dim
o_fine(p) = f^2( concat(feat_mid(p), feat_fine(p)) )
feat_coarse(p) = trilinear_interp(grid_coarse, p)
o_coarse(p) = f^0( feat_coarse(p) )
Color path:
feat_color(p) = trilinear_interp(grid_color, p) -- 32-dim
c(p) = g_omega( feat_color(p) )
Unobserved regions (coarse prior only):
o(p) = f^0( feat_coarse(p) )Coarse and mid levels are first optimized independently to establish a global room shape prior; then all levels participate jointly in the fine mapping optimization. The color decoder is always trained online from scratch for the current scene.
Operator Mechanics
Volumetric Rendering
NICE-SLAM uses NeRF-style volume rendering to produce rendered depth and color for each camera ray r(t) = o + t*d. For the mathematical foundations of volume rendering, see Volume Rendering, Radiance Fields, and Gaussian Splatting.
Ray sampling: stratified sampling with N_strat = 32 points plus importance resampling (N_imp = 16 points) guided by the coarse occupancy distribution — 48 total sample points per ray.
Coarse rendering weights (using coarse occupancy o^0):
w_i_c = o^0(p_i) * prod_{j=1}^{i-1} (1 - o^0(p_j))Fine rendering weights (using combined mid+fine occupancy o):
w_i_f = o(p_i) * prod_{j=1}^{i-1} (1 - o(p_j))Rendered depth (both levels):
D_hat_c(r) = sum_i w_i_c * t_i
D_hat_f(r) = sum_i w_i_f * t_iRendered depth variance (used in the tracking loss):
D_hat_c_var(r) = sum_i w_i_c * (t_i - D_hat_c)^2
D_hat_f_var(r) = sum_i w_i_f * (t_i - D_hat_f)^2Rendered color:
I_hat(r) = sum_i w_i_f * c(p_i)All quantities are differentiable with respect to both grid features (via trilinear interpolation) and camera pose (via the ray origin o and direction d). The SE(3) pose Jacobian propagates through the ray generation; for the underlying Lie group math see Lie Groups — SE(3), SO(3), Jacobians.
Loss Functions
Mapping geometric loss (L1 depth, at both coarse and fine levels):
L_g_l = (1/M) * sum_{m=1}^{M} |D_m - D_hat_m_l|, l in {coarse, fine}Mapping photometric loss (L1 color):
L_p = (1/M) * sum_{m=1}^{M} |I_m - I_hat_m|Total mapping loss:
L_mapping = L_g_coarse + L_g_fine + lambda_p * L_pwhere M = number of randomly sampled pixels (rays) per mapping step.
Tracking loss (variance-weighted depth — the key tracking innovation):
L_track = (1/M_t) * sum_{m=1}^{M_t} [
|D_m - D_hat_m_c| / sqrt(D_hat_c_var_m)
+ |D_m - D_hat_m_f| / sqrt(D_hat_f_var_m)
]The variance weighting down-weights rendered depths that are uncertain (thin structures, partially initialized regions), improving tracking stability in partially-mapped scenes. Regions where the map is confidently reconstructed contribute proportionally more to the pose gradient.
Tracking — Pose Optimization
Per-frame camera pose T_t in SE(3) is optimized by gradient descent on L_track with all feature grid values and decoder weights held fixed:
min_{T_t} L_track(T_t; grids fixed, decoders fixed)Adam optimizer; 300 iterations per tracking step. The pose gradient dL/dT_t propagates through ray generation (parameterized by T_t), volumetric rendering, and the depth residual terms.
Mapping — Feature Grid Optimization
Mapping jointly optimizes feature grid values and a window of keyframe poses over K selected keyframes, with the current-frame pose fixed:
min_{grids, {T_k}_{k in KF}} sum_{k in KF} L_mapping(k)Adam optimizer; 60 iterations per mapping event. Mapping is triggered every 5 frames on most datasets. The coarse grid is optimized first (global shape), then mid and fine grids are optimized jointly.
Keyframe Selection
NICE-SLAM uses an overlap-based keyframe selection strategy rather than a fixed temporal window:
- Randomly sample pixels from the current frame.
- Back-project those pixels using the current depth estimate into 3D world coordinates.
- Project the resulting 3D points into each existing keyframe's image plane.
- Measure the fraction of points with valid projections (within image bounds and within depth tolerance) — the overlap ratio.
- Randomly select
K-2keyframes weighted by overlap ratio from those with non-zero overlap. - Add the most recent keyframe and the current frame. Total
Kframes participate in the mapping optimization.
This criterion ensures the active keyframe window always contains frames observing the same scene regions as the current frame — preventing spurious gradients from non-overlapping views and enabling local map consistency without full global bundle adjustment.
Inputs and Outputs
Inputs (per frame):
- RGB image
I_t(H x W x 3) - Depth image
D_t(H x W x 1) from a calibrated RGB-D sensor (RealSense, Kinect, simulated) - Initial pose estimate (identity for frame 0; previous-frame pose for subsequent frames)
Outputs (incremental, per frame):
- Per-frame camera pose
T_t in SE(3) - Updated multi-resolution feature grids encoding scene geometry and appearance
Final outputs (post-sequence):
- Dense reconstructed mesh (marching cubes from the occupancy field)
- Novel-view rendering (depth and color at arbitrary viewpoints)
- Full camera trajectory (saved poses for all frames)
Depth input is mandatory in the original paper. No monocular-only operating mode is provided; the NICER-SLAM follow-up (see Variants section) extends to monocular RGB.
Benchmark Results
Replica Dataset (8 Synthetic Indoor Rooms)
Reconstruction quality — averages over 8 scenes, 5 independent runs each:
| Method | Mem (MB) | Depth L1 (cm) | Accuracy (cm) | Completion (cm) | Comp. Ratio (%) |
|---|---|---|---|---|---|
| TSDF-Fusion | 67.10 | 7.57 | 1.60 | 3.49 | 86.08 |
| iMAP* | 1.04 | 7.64 | 6.95 | 5.33 | 66.60 |
| DI-Fusion | 3.78 | 23.33 | 19.40 | 10.19 | 72.96 |
| NICE-SLAM | 12.02 | 3.53 | 2.85 | 3.00 | 89.33 |
NICE-SLAM halves the iMAP* Depth L1 error (3.53 vs 7.64 cm) and raises completion ratio by ~23 percentage points, while using 5.6x less memory than TSDF-Fusion.
Tracking — ATE RMSE per scene (cm):
| Scene | NICE-SLAM | Vox-Fusion | ESLAM |
|---|---|---|---|
| room-0 | 1.69 | 0.40 | 0.71 |
| room-1 | 2.04 | 0.54 | 0.70 |
| room-2 | 1.55 | 0.54 | 0.52 |
| office-0 | 0.99 | 0.50 | 0.57 |
| office-1 | 0.90 | 0.46 | 0.55 |
| office-2 | 1.39 | 0.75 | 0.58 |
| office-3 | 3.97 | 0.50 | 0.72 |
| office-4 | 3.08 | 0.60 | 0.63 |
| Average | ~1.95 | ~0.54 | 0.63 |
Sources: Vox-Fusion paper (arXiv:2210.15858) Table 1; ESLAM paper (arXiv:2211.11704). Per-scene values from follow-up papers' comparison tables; small numerical differences vs the NICE-SLAM paper's own table may arise from re-implementation seeds.
TUM RGB-D
ATE RMSE (cm) — from NICE-SLAM Table 2:
| Sequence | iMAP | iMAP* | DI-Fusion | NICE-SLAM | BAD-SLAM | ORB-SLAM2 |
|---|---|---|---|---|---|---|
| fr1/desk | 4.9 | 7.2 | 4.4 | 2.7 | 1.7 | 1.6 |
| fr2/xyz | 2.0 | 2.1 | 2.3 | 1.8 | 1.1 | 0.4 |
| fr3/office | 5.8 | 9.0 | 15.6 | 3.0 | 1.7 | 1.0 |
NICE-SLAM outperforms both iMAP variants and DI-Fusion but trails classical methods (ORB-SLAM2, BAD-SLAM). This gap between neural-implicit and classical SLAM tracking persisted until loop-closure methods appeared (GO-SLAM, Loopy-SLAM).
ScanNet (6 Large-Scale Real Indoor Scenes)
ATE RMSE (cm) — from NICE-SLAM Table 3:
| Scene | iMAP* | DI-Fusion | NICE-SLAM |
|---|---|---|---|
| scene0000 | 55.95 | 62.99 | 8.64 |
| scene0059 | 32.06 | 128.00 | 12.25 |
| scene0106 | 17.50 | 18.50 | 8.09 |
| scene0169 | 70.51 | 75.80 | 10.28 |
| scene0181 | 32.10 | 87.88 | 12.93 |
| scene0207 | 11.91 | 100.19 | 5.59 |
| Average | 36.67 | 78.89 | 9.63 |
The 3.8x improvement over iMAP* on ScanNet is the most dramatic result — demonstrating that hierarchical local updates enable scalability that a single global MLP cannot achieve. DI-Fusion collapses on two of the six scenes.
Co-Fusion Dataset
| Method | ATE RMSE (cm) |
|---|---|
| iMAP* | 7.8 |
| NICE-SLAM | 1.6 |
Results on this dataset with dynamic objects are better than iMAP because NICE-SLAM's local-update structure limits the geographic spread of contamination, though explicit dynamic-object handling is absent.
Speed and Compute
Timing (from NICE-SLAM Table 4)
| Metric | iMAP | NICE-SLAM | Factor |
|---|---|---|---|
| FLOPs per point query (x10^3) | 443.91 | 104.16 | 4.3x fewer |
| Tracking (ms per frame) | 101 | 47 | 2.1x faster |
| Mapping (ms per keyframe) | 448 | 130 | 3.4x faster |
NICE-SLAM's FLOPs per point query are constant regardless of scene size because it queries a fixed number of points from local grid cells. iMAP's single MLP may require more parameters as the scene grows to maintain capacity, increasing per-query cost.
Overall sequence speed: NICE-SLAM processes a full Replica room sequence in a few minutes on a modern GPU. It is not real-time; it is appropriate for offline mapping or post-processing only. GPU memory: approximately 5 GB per room on Replica.
Important clarification on the "10x improvement" figure: Some sources describe NICE-SLAM as achieving "10x improvement over iMAP." This refers to the ATE RMSE accuracy improvement (~10x better on Replica: ~1.95 cm vs ~18.34 cm for iMAP*) — NOT to wall-clock speed. The actual mapping speedup over iMAP is 3.4x and tracking speedup is 2.1x. ESLAM later achieves approximately 10x wall-clock speedup over NICE-SLAM itself.
Speed in Lineage Context
| Method | Tracking | Mapping | Overall | Notes |
|---|---|---|---|---|
| iMAP | ~10 Hz | ~2 Hz | Sub-real-time | Global MLP, slow |
| NICE-SLAM | ~21 Hz theoretical | 130 ms/KF | Minutes/sequence | Not real-time end-to-end |
| Vox-Fusion | ~5 Hz | ~2 Hz | Near real-time | Sparse allocation |
| ESLAM | — | — | ~10x faster than NICE-SLAM | Tri-plane, no pre-training |
| Co-SLAM | ~17 Hz | ~12 Hz | ~10 Hz overall | Hash grid, global BA |
| SplaTAM | — | — | 0.3–0.5 FPS (SLAM loop) | 3DGS rendering, fast render |
Lineage and Context
NICE-SLAM is the second method in the neural-implicit-SLAM lineage, immediately following iMAP:
2021 — iMAP (Sucar et al., ICCV 2021) [arXiv:2103.12352]
Single global MLP. Proof of concept.
Catastrophic forgetting; fails at scale.
Replica Depth L1 ~7.64 cm; ATE ~18.34 cm avg.
2022 — NICE-SLAM (Zhu et al., CVPR 2022) [arXiv:2112.12130] <- THIS PAGE
Hierarchical feature grids (4 levels) + pre-trained decoders.
Local updates eliminate catastrophic forgetting.
Scales to ScanNet (9.63 cm avg ATE vs 36.67 cm iMAP*).
2022 — Vox-Fusion (Yang et al., ISMAR 2022) [arXiv:2210.15858]
Sparse dynamic voxel allocation (no pre-allocated dense grid).
SDF representation instead of occupancy.
Better per-scene Replica ATE (~0.54 cm avg).
2023 — ESLAM (Johari et al., CVPR 2023) [arXiv:2211.11704]
Tri-plane SDF; no pre-training; ~10x faster than NICE-SLAM.
Replica ATE avg ~0.63 cm.
2023 — Co-SLAM (Wang et al., CVPR 2023) [arXiv:2304.14377]
Hash grid + one-blob hybrid; global BA; ~10 Hz real-time.
2023 — GO-SLAM (Zhang et al., ICCV 2023) [arXiv:2309.02436]
Loop closure + Instant-NGP; Replica ATE ~0.35 cm.
2023 — Point-SLAM (Sandstrom et al., ICCV 2023) [arXiv:2304.04278]
Neural point cloud; adaptive density; Replica Depth L1 0.44 cm.
2024 — Loopy-SLAM (Liso et al., CVPR 2024) [arXiv:2402.09944]
Sub-map loop closure; apex of NeRF-SLAM lineage.
Replica ATE 0.29 cm, Depth L1 0.35 cm.
2024+ — Field pivots to 3DGS-SLAM:
SplaTAM, MonoGS, GS-SLAM (all CVPR 2024)
See [Splat-SLAM](./splat-slam.md) and [GS-SLAM and MonoGS](./gs-slam-monogs.md)NICE-SLAM's historical position: the first scalable neural-implicit SLAM. Every successor in this lineage (Vox-Fusion, ESLAM, Co-SLAM, Point-SLAM, Loopy-SLAM) is a direct architectural response to NICE-SLAM's limitations — speed, dense pre-allocation, tracking accuracy. Understanding NICE-SLAM is prerequisite to evaluating the improvement claims of all these systems.
See the NeRF-SLAM family overview for the full lineage with per-method technical summaries.
Cross-Method Benchmark Summary
Replica — Tracking ATE RMSE (cm), average over 8 scenes:
| Method | ATE avg (cm) | Year | Representation |
|---|---|---|---|
| iMAP* | ~18.34 | 2021 | Single global MLP |
| NICE-SLAM | ~1.95 | 2022 | Hierarchical feature grids |
| Vox-Fusion | ~0.54 | 2022 | Sparse voxel SDF |
| ESLAM | ~0.63 | 2023 | Tri-plane SDF |
| Co-SLAM | ~0.78 | 2023 | Hash + one-blob, global BA |
| GO-SLAM | ~0.35 | 2023 | Loop closure + Instant-NGP |
| Point-SLAM | ~0.52 | 2023 | Neural point cloud |
| Loopy-SLAM | ~0.29 | 2024 | Sub-map loop closure |
| SplaTAM (3DGS) | ~0.31 | 2024 | 3DGS representation |
Replica — Reconstruction Depth L1 (cm):
| Method | Depth L1 avg (cm) |
|---|---|
| iMAP* | 7.64 |
| NICE-SLAM | 3.53 |
| ESLAM | 1.18 |
| Vox-Fusion | ~0.54 |
| Co-SLAM | ~0.50 |
| GO-SLAM | ~0.41 |
| Point-SLAM | 0.44 |
| Loopy-SLAM | 0.35 |
Sources: NICE-SLAM (arXiv:2112.12130); Loopy-SLAM comparison table (arXiv:2402.09944); Vox-Fusion (arXiv:2210.15858); ESLAM (arXiv:2211.11704).
Variants and Follow-Ups
NICER-SLAM (3DV 2024, Best Paper Honorable Mention)
Citation: Zihan Zhu, Songyou Peng, Viktor Larsson, Zhaopeng Cui, Martin R. Oswald, Marc Pollefeys. "NICER-SLAM: Neural Implicit Scene Encoding for RGB SLAM." 3DV 2024. arXiv:2302.03594. GitHub: https://github.com/cvg/nicer-slam. Project: https://nicer-slam.github.io/
NICER-SLAM drops the depth sensor requirement entirely. It is a monocular RGB SLAM that retains the hierarchical neural implicit map from NICE-SLAM but adds:
- Monocular depth estimation from the implicit map itself (rendered depth as self-supervision).
- Monocular geometric cues from off-the-shelf depth and normal estimators to bootstrap mapping.
- An optical flow warp loss enforcing photometric consistency across frames.
The system simultaneously optimizes camera poses and the neural scene representation from pure RGB video. Unlike the original NICE-SLAM evaluation (which uses a culled mesh, removing faces outside the camera frustum at training time), NICER-SLAM evaluates on the original unculled mesh — demonstrating geometry extrapolation beyond directly observed regions.
Adaptive Feature Grids (arXiv:2306.02395, 2023)
Citation: Ganlin Zhang, Ling Peng. "NICE-SLAM with Adaptive Feature Grids." arXiv:2306.02395. GitHub: https://github.com/zhangganlin/NICE-SLAM-with-Adaptive-Feature-Grids
Addresses NICE-SLAM's most critical scalability limit: the dense pre-allocated feature grids. Memory scales as O((S/r)^3) for scene size S and voxel side r, making large or outdoor scenes infeasible. This work incorporates voxel hashing (inspired by InfiniTAM and Vox-Fusion) into the NICE-SLAM framework — allocating feature vectors dynamically only near observed surfaces. Memory footprint is substantially reduced with reconstruction quality comparable to the original.
EvenNICER-SLAM (arXiv:2410.03812, 2024)
Extends the NICE-SLAM architecture with event camera input, replacing or augmenting the RGB-D stream with event data for high-dynamic-range and fast-motion scenes. Research-stage, not evaluated at production scale.
Strengths
Scalability beyond iMAP. Hierarchical local updates enable room-scale and multi-room scenes where iMAP collapses. The ScanNet 3.8x ATE improvement over iMAP* (9.63 vs 36.67 cm average) is the key proof.
Dense reconstruction quality. Depth L1 of 3.53 cm on Replica at 12 MB representation size outperforms TSDF-Fusion (7.57 cm) while using 5.6x less memory. The implicit surface is smooth and geometrically complete.
Pre-trained geometric priors accelerate convergence. Coarse and mid decoders initialized from ConvONet provide a plausible geometry starting point from the first frame. Tracking recovers from frame loss within ~300 iterations using the coarse-level prediction — the coarse rendering provides a usable prediction even in partially mapped regions.
Compact representation. 12.02 MB for a full Replica room, versus tens to hundreds of MB for classical dense voxel maps at equivalent detail.
Differentiable end-to-end. Pose Jacobians flow via automatic differentiation through the trilinear interpolation, volumetric rendering, and depth residual chain. No separate ICP or feature-matching step is needed.
Open-source and reproducible. Clean PyTorch implementation at
cvg/nice-slam(Apache-2.0) with download scripts for all standard datasets, evaluation scripts, and an iMAP* comparison implementation.
Failure Modes
Not real-time. Mapping takes minutes per Replica sequence. Tracking at ~47 ms/frame (~21 Hz theoretical) is near real-time, but the interleaved mapping step makes overall throughput well below 1 Hz for most sequences. Not viable for any online loop in a vehicle stack.
Dense pre-allocation memory explosion. Grid memory scales as O((S/r)^3). Large outdoor or multi-building scenes are infeasible without the Adaptive Feature Grids extension (arXiv:2306.02395). A 100 m outdoor scene at 16 cm voxel resolution would require prohibitive memory in the standard implementation.
RGB-D required in the original paper. No monocular mode. The depth sensor is mandatory. NICER-SLAM addresses this, but at reduced accuracy. Commodity RGB-D sensors calibrated for indoor use are impractical for the range requirements of outdoor AV environments (>50 m for apron scenes).
No loop closure. Drift accumulates on long sequences without a place-recognition backend. Large-scale trajectories will diverge. Loop closure arrived late in the lineage (GO-SLAM, Loopy-SLAM).
Dynamic scenes. Moving people, vehicles, or equipment cause persistent ghost artifacts in the neural map. NICE-SLAM has no masking or motion-segmentation capability; dynamic regions corrupt the feature grids. The local-update structure limits geographic spread of contamination (demonstrated by the 1.6 cm ATE on the Co-Fusion dynamic dataset vs iMAP's 7.8 cm), but artifacts remain in the mesh.
Accuracy trailed by all successors. Vox-Fusion's Replica ATE (~0.54 cm avg) is 3.6x better. ESLAM, Co-SLAM, and Point-SLAM widen this gap further. By 2024, NICE-SLAM is a historical baseline, not a competitive system.
Textureless surfaces. The photometric loss L_p provides no gradient signal on uniform walls, ceilings, or floors. The depth loss alone must carry reconstruction in these areas.
Fast motion and motion blur. At high angular velocity, the 300-iteration tracking optimizer can diverge before converging on the true pose. The variance-weighted tracking loss partially mitigates this by ignoring uncertain render regions, but severe blur is unresolvable.
Transparent and reflective surfaces. NeRF's volumetric occupancy model is ambiguous for glass and mirrors. Tracking fails at transparent surfaces — common in indoor AV environments (office partitions, glass doors).
Domain Fit
| Domain | Fit | Key Notes |
|---|---|---|
| Indoor rooms / labs | Good (research) | Standard benchmark setting; Replica/TUM/ScanNet representative |
| Building-scale indoor | Conditional | Memory and loop-closure limits apply; use GO-SLAM or Loopy-SLAM |
| Airside hangar / terminal (offline appearance) | Conditional | Appearance digital twin only; needs LiDAR primary pose source |
| Airside apron (primary SLAM) | Not suitable | Low texture, dynamic GSE, no LiDAR, range limits, no loop closure |
| Road AV (offline appearance layer) | Research | Camera-only; superseded by 3DGS offline reconstruction for this role |
| Road AV (online primary localization) | Not suitable | Not real-time; no LiDAR; not safety-critical ready |
| Warehouse / port (short-range indoor) | Conditional | Viable in static, textured areas; dynamic equipment needs masking |
| Agriculture / construction | Not suitable | Outdoor scale, weather, dynamic machinery, vegetation |
| Simulation asset generation (indoor) | Good (research) | View synthesis and digital twin from controlled captures |
Aggregated-Map Suitability — Honest Assessment
NICE-SLAM is an RGB-D visual SLAM system. It does not consume LiDAR data. Its role in any aggregated-map or AV-stack context must be carefully scoped.
What NICE-SLAM can contribute:
- Dense visual appearance capture of fixed structures (terminal facades, jetbridge geometry, stand markings) for cross-checking or texture-mapping a LiDAR-derived geometry mesh.
- Novel-view rendering for map inspection or simulation-asset generation (offline, controlled capture).
- As a research baseline: all 3DGS-SLAM papers benchmark against NICE-SLAM. Reading those papers requires understanding NICE-SLAM.
What NICE-SLAM cannot do in the airside or outdoor AV context:
- Does not process LiDAR point clouds. The LiDAR-native counterparts are PIN-SLAM (arXiv:2401.09101, TRO 2024) and 4DNDF (iter-32) for neural implicit approaches, or KISS-SLAM and FAST-LIO2 for classical-production LiDAR methods.
- Airside depth perception at range (>50 m) is impractical with commodity RGB-D sensors.
- No loop closure — will drift over long trajectories typical of ramp survey drives.
- Fails in adverse weather (fog, rain, direct sunlight) where the photometric loss provides unreliable signal.
- Not real-time — inappropriate for the on-vehicle SLAM loop.
Has NICE-SLAM been superseded for its own use case? Yes. For camera-based dense SLAM with an RGB-D sensor, 3DGS-SLAM systems — SplaTAM, MonoGS, GS-SLAM (all CVPR 2024) — have largely superseded NICE-SLAM as of 2024:
- SplaTAM achieves approximately 10 dB higher PSNR than NICE-SLAM on Replica.
- 3DGS rasterization is approximately 100x faster than NeRF-style volume rendering.
- Gaussian primitives are explicit and more tractable for geometric analysis and loop correction.
For new camera-only dense SLAM work, choose a 3DGS-SLAM method (see Splat-SLAM and GS-SLAM and MonoGS). For the appearance-layer role over a LiDAR geometry map, offline 3D Gaussian Splatting reconstruction (nerfstudio, OpenSplat) is simpler and higher quality than live SLAM. See Aggregated Map Semantic Segmentation for downstream use of such appearance layers.
NICE-SLAM is a historically important method — the paper that defined the hierarchical feature-grid paradigm adopted by all successors — and a required benchmark baseline, but not a deployment candidate for new AV stacks.
Implementation Notes
- Official repository:
cvg/nice-slam(Apache-2.0). Includes download scripts for Replica, ScanNet, TUM RGB-D, Co-Fusion, and a multi-room apartment demo sequence. Also includes an iMAP* re-implementation for direct comparison. - Dependencies: Python / PyTorch with standard geometry and visualization tooling. GPU support is required for meaningful performance; the system was designed for an RTX-class card with approximately 5 GB VRAM per Replica sequence.
- Scene bounds: NICE-SLAM requires the scene bounding box to be specified before the run (to allocate the dense feature grids). This is a significant operational difference from sparse methods (Vox-Fusion, Co-SLAM) that allocate on demand. Budget the grid memory against available VRAM before starting.
- Voxel resolution selection: the standard Replica and ScanNet resolutions (2 m / 32 cm / 16 cm / 16 cm) are scene-size-dependent. Smaller indoor rooms should use the TUM settings (2 m / 16 cm / 8 cm / 8 cm) to avoid under-resolved grids.
- Evaluation protocol: the standard NICE-SLAM paper evaluates reconstruction on a culled mesh (faces outside the training frustum are removed). NICER-SLAM and some follow-up papers evaluate on the unculled mesh — report which protocol is used when comparing numbers across papers.
- Comparison table consistency: per-scene Replica ATE numbers vary across comparison papers (Vox-Fusion, ESLAM, SplaTAM each re-run NICE-SLAM with slightly different seeds or configs). The authoritative source for NICE-SLAM's own numbers is its CVPR 2022 paper and arXiv:2112.12130, not re-reported values.
- Integration with ROS2: the repository is not a production robotics stack. It requires integration work for ROS2 sensor bridges, health monitoring, dynamic object masking, deterministic logging, and deployment supervision.
- Airside survey use: if collecting an appearance digital twin of an indoor airside zone, run NICE-SLAM offline against a controlled RGB-D capture with a separate metric pose source (LiDAR-inertial or RTK). Validate geometry against raw depth or LiDAR before any operational claim. Do not feed NICE-SLAM output directly to a safety-critical navigation or clearance system.
- NICER-SLAM: for monocular experiments, use
cvg/nicer-slamrather than NICE-SLAM proper. Be aware that NICER-SLAM evaluates on unculled meshes and its numbers are not directly comparable to the culled-mesh NICE-SLAM tables. - Adaptive grids extension: for larger scenes,
zhangganlin/NICE-SLAM-with-Adaptive-Feature-Grids(arXiv:2306.02395) provides voxel-hashing sparse allocation on top of the standard NICE-SLAM code. Treat this as an ablation engineering contribution, not a maintained production fork.
Sources
| Resource | URL |
|---|---|
| NICE-SLAM arXiv | https://arxiv.org/abs/2112.12130 |
| NICE-SLAM CVPR open access | https://openaccess.thecvf.com/content/CVPR2022/html/Zhu_NICE-SLAM_Neural_Implicit_Scalable_Encoding_for_SLAM_CVPR_2022_paper.html |
| NICE-SLAM project page | https://pengsongyou.github.io/nice-slam |
| NICE-SLAM GitHub | https://github.com/cvg/nice-slam |
| NICER-SLAM arXiv | https://arxiv.org/abs/2302.03594 |
| NICER-SLAM GitHub | https://github.com/cvg/nicer-slam |
| NICER-SLAM project page | https://nicer-slam.github.io/ |
| Adaptive Feature Grids arXiv | https://arxiv.org/abs/2306.02395 |
| Adaptive Feature Grids GitHub | https://github.com/zhangganlin/NICE-SLAM-with-Adaptive-Feature-Grids |
| iMAP arXiv | https://arxiv.org/abs/2103.12352 |
| Vox-Fusion arXiv | https://arxiv.org/abs/2210.15858 |
| ESLAM arXiv | https://arxiv.org/abs/2211.11704 |
| Co-SLAM arXiv | https://arxiv.org/abs/2304.14377 |
| Loopy-SLAM arXiv | https://arxiv.org/abs/2402.09944 |
| PIN-SLAM arXiv | https://arxiv.org/abs/2401.09101 |
- Local context: NeRF-SLAM family overview — iter-27; full neural-implicit-SLAM lineage with per-method summaries
- Local context: iMAP — single-MLP predecessor; the baseline NICE-SLAM was designed to surpass
- Local context: Co-SLAM and ESLAM — hash-grid and tri-plane successors
- Local context: Splat-SLAM — iter-18; 3DGS-SLAM with loop closure; has superseded NICE-SLAM for appearance-layer use
- Local context: GS-SLAM and MonoGS — iter-26; first-wave 3DGS-SLAM systems at CVPR 2024
- Local context: MASt3R-SLAM — iter-24; feed-forward SLAM successor direction
- Local context: 4DNDF — iter-32; 4D neural distance field for LiDAR-primary neural mapping
- Local context: KISS-SLAM — iter-32; LiDAR-primary production odometry counterpart
- Local context: Aggregated Map Semantic Segmentation — downstream use of appearance maps
- Local context: Feed-Forward 3D Reconstruction and Splatting — iter-16; 3DGS first principles
- Local context: Volume Rendering, Radiance Fields, and Gaussian Splatting — NeRF and volume rendering background
- Local context: Lie Groups — SE(3), SO(3), Jacobians — iter-14; SE(3) pose Jacobian math underlying the tracking optimization