Trifocal tensor

The tensor can also be seen as a collection of three rank-two 3 x 3 matrices ${\displaystyle {\mathbf {T} }_{1},\;{\mathbf {T} }_{2},\;{\mathbf {T} }_{3}}$ known as its correlation slices. Assuming that the projection matrices of three views are ${\displaystyle {\mathbf {P} }=[{\mathbf {I} }\;|\;{\mathbf {0} }]}$, ${\displaystyle {\mathbf {P} }'=[{\mathbf {A} }\;|\;{\mathbf {a} }_{4}]}$ and ${\displaystyle {\mathbf {P} ''}=[{\mathbf {B} }\;|\;{\mathbf {b} }_{4}]}$, the correlation slices of the corresponding tensor can be expressed in closed form as ${\displaystyle {\mathbf {T} }_{i}={\mathbf {a} }_{i}{\mathbf {b} }_{4}^{t}-{\mathbf {a} }_{4}{\mathbf {b} }_{i}^{t},\;i=1\ldots 3}$, where ${\displaystyle {\mathbf {a} }_{i},\;{\mathbf {b} }_{i}}$ are respectively the i^th columns of the camera matrices. In practice, however, the tensor is estimated from point and line matches across the three views.

One of the most important properties of the trifocal tensor is that it gives rise to linear relationships between lines and points in three images. More specifically, for triplets of corresponding points ${\displaystyle {\mathbf {x} }\;\leftrightarrow \;{\mathbf {x} }'\;\leftrightarrow \;{\mathbf {x} }''}$ and any corresponding lines ${\displaystyle {\mathbf {l} }\;\leftrightarrow \;{\mathbf {l} }'\;\leftrightarrow \;{\mathbf {l} }''}$ through them, the following trilinear constraints hold:

${\displaystyle ({\mathbf {l} }^{\prime t}\left[{\mathbf {T} }_{1},\;{\mathbf {T} }_{2},\;{\mathbf {T} }_{3}\right]{\mathbf {l} }'')[{\mathbf {l} }]_{\times }={\mathbf {0} }^{t}}$

${\displaystyle {\mathbf {l} }^{\prime t}\left(\sum _{i}x_{i}{\mathbf {T} }_{i}\right){\mathbf {l} }''=0}$

${\displaystyle {\mathbf {l} }^{\prime t}\left(\sum _{i}x_{i}{\mathbf {T} }_{i}\right)[{\mathbf {x} }'']_{\times }={\mathbf {0} }^{t}}$

${\displaystyle [{\mathbf {x} }']_{\times }\left(\sum _{i}x_{i}{\mathbf {T} }_{i}\right){\mathbf {l} }''={\mathbf {0} }}$

${\displaystyle [{\mathbf {x} }']_{\times }\left(\sum _{i}x_{i}{\mathbf {T} }_{i}\right)[{\mathbf {x} }'']_{\times }={\mathbf {0} }_{3\times 3}}$

where ${\displaystyle [\cdot ]_{\times }}$ denotes the skew-symmetric cross product matrix.

Given the trifocal tensor of three views and a pair of matched points in two views, it is possible to determine the location of the point in the third view without any further information. This is known as point transfer and a similar result holds for lines and conics. For general curves, the transfer can be realized through a local differential curve model of osculating circles (i.e., curvature), which can then be transferred as conics.[2] The transfer of third-order models reflecting space torsion using calibrated trifocal tensors have been studied,[3] but remains an open problem for uncalibrated trifocal tensors.

• Hartley, Richard I. (1997). "Lines and Points in Three Views and the Trifocal Tensor". International Journal of Computer Vision. 22 (2): 125–140. doi:10.1023/A:1007936012022. S2CID 8979544.
• Torr, P. H. S.; Zisserman, A. (1997). "Robust Parameterization and Computation of the Trifocal Tensor". Image and Vision Computing. 15 (8): 591–607. CiteSeerX 10.1.1.41.3172. doi:10.1016/S0262-8856(97)00010-3.

• Visualization of trifocal geometry (originally by Sylvain Bougnoux of INRIA Robotvis, requires Java)