We emphasize the importance of bridges between statistics, mechanics, and geometry. We develop and employ links between pencils of quadrics, moments of inertia, and linear and orthogonal regressions. For a given system of points in $R^k$ representing a sample of a full rank, we construct a pencil of confocal quadrics which appears to be a useful geometric tool to study the data. Some of the obtained results can be seen as generalizations of classical results of Pearson on orthogonal regression. Applications include statistics of errors-in-variables models (EIV) and restricted regressions, both ordinary and orthogonal ones. For the latter, a new formula for test statistic is derived. The developed methods and results are illustrated in natural statistics examples. The talk is based on joint results with Borislav Gajić.