Tom from the PCKicks Computer Help Force explains how different partitioning and filesystem formatting can affect how much space you are wasting.
I am in the process of massively downloading very detailed maps onto my Pocket PC, and for the past two days I’ve been kept up all night while my computer was downloading and copying the files on to my memory card.
Well anyways, I thought it would be interesting to do a write up about disk space.
It occured to me that not many people know of the limitations of disk space. What I am stating is everything has an extra charge to it including disk space. (taxes, hidden fees, and etc)
Now in a nutshell in order to store files your computer needs something called a “file system.” And a file system is made up of something called “allocation units”. This basically means your logical file system on your hard drive, floppy disk, and USB flash drives is made up of “boxes” or units in which they store data in chunks.
With that said, the theory of data storage on a file system is complex and that was my best way of putting it into laymen terms.
But just incase if anyone didn’t get that: Take of instance your table surface, in order to put things on it you need boxes. Now picture your table surface completely populated with those boxes. Ala allocation units.
Now here comes the bone shattering truth of computing disk space.
Take for instance you have 2,109 1K text files. (Which is roughly the eqvenilent to 1000 characters in one text file)
You would think”Ah yes! that takes 2109 kilobytes on my hard drive space right? ” Nope
What actually happens is you’ll end up inadvertently wasting disk space depending on your partitioning scheme.
Below is an example of this phenomenon:
One 2GB partition has been formatted with 512 clusters (allocation units) The other 2GB partition with 64KB clusters (allocation units)
I guess pictures say more than words so take a look at the picture below:
A USB flash drive in the shape of a piece of ikura (salmon roe) sushi. Photo taken by Tokugawapants using Konica-Minolta Maxxum 5d with Minolta AF 100mm Macro lens. (Photo credit: Wikipedia)
This image shows the drastic difference between 512 byte clusters vs 64K clusters. The window on the left was formatted with 512b clusters, and the right with 64KB clusters.
Now it’s all fine and dandy when the dialog box shows “Size: 2.05KB” But what most people fail to understand and see is that the data actually took up to 131 MB to store on the hard drive!
Now you ask why is this happening? Well it all goes back to the whole “box” idea.
Each piece of data has to be stored in its own “box” or allocation unit. So when you have larger “boxes” the physical space is wasted despite the box is not filled to the brim with “stuff” or in this case data.
With this setup you end up wasting a large amount of space storing a few small files on the hard drive.
Also keep in mind that the bigger your clusters are the more space in one single partition you can have. For example 512byte clusters won’t support an 8 GB partition.
Is there something you can do to get away from this? Well, yes and ….. no.
You could try using a different file system. One that is optimized for this usage, but then you end up losing compatibility with other devices or operating systems. Or it is also possible to create smaller partitions out of a bigger one for example 100 2GB partitions of a 200 GB hard drive, but again it is not practical.
So in short, no there isn’t a real way where you can avoid this from happening. Like buying items or paying for your cell phone bill there is always an extra penalty.
Here is also an interesting limitation:
Did you know you cannot have a file with the same name of a directory? This also goes vise versa.
With this space limitation being big deal I think is pointless to complain about memory cards being “falsely advertised”. I.E. The whole KB rounding and all. :^)
PCKicks Computer Help Force