CoDimRecon: reconstructing scenes that can be simulated
Curves, shells, and solids give deformable objects different physical representations.
CoDimRecon scene reconstruction and deformable interactions. Credit: CoDimRecon authors. Original image ↗
CoDimRecon reconstructs editable, simulation-ready scenes from multiple RGB views. It includes rigid, articulated, and deformable objects, then tests their behavior through simulated robot interactions.
Choose a physical representation before judging appearance
A telephone cord, a plastic bag, and a cushion may all look detailed in a mesh viewer, but they need different descriptions to simulate deformation. A curve captures the long, thin structure of a cord; a shell captures a sheet; a volume captures a solid interior.
CoDimRecon organizes reconstruction around those distinctions. This is useful because the question is ultimately behavioral: will the reconstructed object bend, collide, and move appropriately in the intended interaction? The project’s physics examples connect the selected geometry with later simulation tests.
Represent the object you have
| Object type | Representation | Example |
|---|---|---|
| Curve | Centerline with radius | Telephone cord |
| Surface | Shell with thickness | Paper or plastic bag |
| Volume | Watertight solid | Chair cushion |
The distinction matters: resolving a thin sheet as a full volume can introduce unnecessary complexity. The reconstruction target should fit the intended solver.
Test behavior after geometry
The project describes an agent-guided revision loop. Failures can lead to changes in motion, geometry, numerical settings, or the material model. Its paper-folding example illustrates a behavioral mismatch: a numerically valid elastic sheet springs back when a lasting crease is expected.
Work through a material assumption
An elastic model can explain a simulation that springs back after being released. A simple one-dimensional spring illustrates the distinction. With stiffness k and displacement x, the restoring force magnitude is:
For an illustrative k = 100 newtons per metre and x = 0.02 metres, the force magnitude is two newtons. The chart varies displacement for that same hypothetical stiffness. This is a teaching model, not CoDimRecon's fitted parameters, a calibrated paper model, or a measurement of a reconstructed object.
A spring returns toward its original equilibrium when the external load is removed. A permanent paper crease requires behavior that retains a changed state. Making an elastic mesh look more like paper cannot by itself create that material response. The representation and constitutive assumptions have to express the behavior required by the task. A stable numerical result can therefore be physically unsuitable even when the simulation does not crash.
Keep geometry, material, and numerics separate
Imagine a sheet that passes through a gripper, deforms too little, or springs back after folding. These are different observations. Penetration may concern collision geometry, placement, contact handling, or numerical resolution. Excess stiffness may concern the material parameters or scale. Spring-back may reflect an elastic model where a persistent crease is needed. Diagnose the visible failure before changing every parameter at once.
A fitted parameter should travel with the interaction used to fit it. If an effective stiffness makes one compression example look plausible, it has not automatically been measured as the original object's material constant. Retain the chosen solver, geometry scale, boundary conditions, and observed motion. Those details determine which behaviors the fit supports.
For cords, shells, and volumes, distinguish the representation needed by the simulator from the visible surface used for rendering. A watertight cushion and a thin bag do not need the same interior model. A useful reconstruction provides an object that behaves appropriately in a specified interaction, with its uncertainty and assumptions visible. The agent-guided revision loop is most informative when a reader can follow the diagnosed failure, the proposed change, and the behavioral evidence after that change.
An elastic assumption predicts spring-back
- Select representation
- Simulate an interaction
- Diagnose the mismatch
- Revise the relevant assumption
Focus the graph and use arrow keys, or tap a point, to inspect values. The complete data table is available below.
View data table
| Displacement (cm) | Restoring force (N) |
|---|---|
| 0 | 0 |
| 1 | 1 |
| 2 | 2 |
| 3 | 3 |
| 4 | 4 |
| 5 | 5 |
Use an interaction as the acceptance test
Start with an action whose outcome is visible: folding a sheet, compressing a cushion, or moving a drawer. Compare geometry before the action and behavior during it. If the simulation fails, identify whether the problem comes from shape, placement, constraints, material assumptions, or numerical settings.
The paper-folding example is instructive. An elastic sheet that returns to its starting shape may be mathematically valid for the selected model but wrong for paper that should retain a crease. Improving only the render would not fix that mismatch.
Preserve the distinction between fitted and measured values
An effective simulation parameter describes behavior within a chosen model. It is not automatically a measured property of the original material. Record the assumed material model and the interactions used to revise it so that another reviewer can understand what the reconstructed scene supports.
The site includes an interactive demonstration and videos; code is marked as coming soon at our review. Use those assets to inspect the proposal, and wait for the actual implementation before offering installation instructions. We have not reconstructed the examples independently. A useful first reading is to trace one object from input views through representation choice to the behavioral test that required a revision.
