Reducing IoT Data for Highly Efficient Cloud Storage

The 16th International Conference on Ambient Systems, Networks and Technologies (ANT) April 22-24, 2025, Patras, Greece

We focus on the very important problem of managing IoT data. We consider the data gathering process that yields big data intended for CDN/cloud storage. We aim to reduce big data into small data to efficiently exploit available storage without compromising their usability and interpretation. This reduction process is to be performed at the edge of the infrastructure (IoT edge devices, CDN edge servers) in a computationally acceptable way. Therefore, we employ reservoir sampling, a method that stochastically samples data and derives synopses that are finally pushed and maintained in the available storage capability. We establish a scheme that continuously compares stream statistics with those of the accumulated synopsis (buffer) and determines the proper time instance to push the synopsis to the backend/cloud. We assess the performance of our scheme using real IoT data from the shipping industry.
Our histogram comparisons between the actual stream and the progressive accumulation of synopses indicate very small deviations while a significant compress (volume reduction) benefit is clearly obtained.