Regularized inversion of flow size distribution

Abstract

In this paper, we revisit the estimation of the size distribution of packet flows in Internet traffic through an inversion approach for several packet sampling schemes which are based on probabilistic sampling (PS). We first study the statistical properties of the previously introduced inversion estimator in its general form and make connections to the singular value decomposition. This motivates the use of a regularization technique in the estimation of the flow size distribution. More specifically, a penalized weighted least square approach is proposed. We compare theoretically the penalized estimator under simplified assumptions against the (non-penalized) inversion approach in order to explain differences in their statistical behaviors. A data study with two real traces shows that the proposed penalized estimator outperforms the inversion estimator for all sampling schemes, corroborating the theoretical analysis. This work reveals that the simplest sampling schemes based on PS, that do not work with small sampling probabilities under the inversion approach, can be used with the penalized approach. Furthermore, the penalized approach allows considering smaller packet sampling rates for all the other sampling schemes.