Detecting filesystems at startup problem

[kata23]

Hi Lightning UK,

 

I found an error how ImgBurn detects filesystems.

It was checked in v2.4.2.0 and it is also in v2.4.3.0.

Upon program start the ImgBurn Log window displays a warning:

W 13:35:29 Drive X:\ (FAT) does not support single files > 4 GB in size.

 

However this X:\ drive is not FAT but NSS (NetWare). This can handle even 8TB files in size (since 1998).

I know that there are some libraries programmers use, which can't detect NSS filesystems right, it is possible that you used also such a library.

 

Can you fix that bug, please?

 

If you need a system to test I can help with that too.

 

Thanks.

[LIGHTNING UK!]

This is done via the Windows API (GetVolumeInformation), there is nothing for me to fix so to speak.

[kata23]

I understand that. So you get FAT as a result on the File System Type when using GetVolumeInformation function?

GetVolumeInformation doesn't tell you that the max. file size is 4GB. That is the (wrong) assumption made by ImgBurn after it receives FAT as an answer from the Windows API, am I right?

 

(I found the same problem in BitComet too. It was easier for me to switch to uTorrent, then using their support system.)

 

I try to be more specific this time:

I know that the windows API tells you the file system type, which probably the same answer for NSS or for FAT. (This is windows, which is commonly known not perfectly programmed.)

I'm looking for a solution, where ImgBurn could decide if a volume can handle over 4GB files not based on that function.

Any solution for that?

(I'm willing to help testing it, and if it works I'll help to test Unix volumes mounted through NFS and recognised as FAT volumes's problem too.)

[LIGHTNING UK!]

Yes, if that function says the file systems used by the volume is FAT or FAT32, ImgBurn displays the message saying it can't handle single files > 4GB in size (because they can't!).

 

I don't think it's fair to say ImgBurn makes the wrong assumption, it's entirely correct based on the information is has received.

 

The only test I know of would be to attempt to make a file of that size via createfile, setfilepointer and setendoffile. That's quite expensive on proper fat/fat32 drives though... although maybe not if it's just going to fail rather than actually try.

 

I assume your X: drive is a remote one yeah? (sorry, I know nothing about netware)

 

I also assume you can make (and have made) files > 4gb on that drive from your machine yeah?

[LIGHTNING UK!]

Does this work?

 

fsutil file createnew X:\testfile.bin 4294967296

[kata23]

We have made files > 4GB and > 1TB too on NSS, it works even from windows natively via CIFS protokoll.

 

And yes X:\ is a remote drive, Netware is a file server operating system, it can't run ImgBurn.

 

fsutil file createnew X:\testfile.bin 4294967296 works too, altough it takes around a minute to finish (i'm on a Gbit network).

 

The createfile idea sounds good, as far as I know uTorrent uses sparse files to detect volumes, which can be handled by both NTFS and NSS too and it doesn't slow down the proggy. But I'm not a programmer since many years, so it is an idea only.

[LIGHTNING UK!]

I know what Netware is, I just didn't know if you could take whatever an NSS formatted drive is out of the box and plug it in elsewhere.

 

Ah see now if fsutil takes all that time it's probably not going to be any good me using createfile as I'll run into the same problem - like I said, it's an expensive operation! (on anything other than NTFS so it would seem)

 

Thinking about it, what happens if you read a 3.999gb disc to your X: drive using ImgBurn?

 

Does it take a while making the image file right at the start? (they're allocated in the same way I'd test for the real max file size - which is basically making a sparse file)

 

The statusbar would say 'Allocating File Data Storage Area...' during this time.

[kata23]

You can put an NSS drive into an XP machine, however you can't read it trough explorer, but there are tools, like Nemo Commander, which can read NSS.

 

I've created an 3.6GB .iso file, mounted with Daemon Tools and used ImgBurn "Create image file from disk" command to create an image from it to X:\.

 

It displayed the "'Allocating File Data Storage Area.." message as you expected in the status bar for over a minute.

 

So how does uTorrent does it? When I start a torrent >4GB uTorrent creates the (sparse) file in a split second.

[blutach]

AFAIK, utorrent creates a 0 byte file and just adds to it as the bits come in.

 

Regards

[kata23]

blutach:

I installed uTorrent just to check it.

I opened a 6GB file, and it created it in 2 seconds.

You can test it too, it took only 2 minutes to download, install, search, create, and post this reply.

[LIGHTNING UK!]

If uTorrent works exactly how you want ImgBurn to (and it really does allocate the full 4gb straight off), feel free to ask them how they did it and report back.

 

For now I'll just allow you to override the reported FS limitation once you set file splitting to 'None'.

[blutach]

It didn't create the full 6Gb in 2 seconds - it may have allocated space for it, however.

 

Regards

[kata23]

If found this in the uTorrent forum:

 

"When allocating large files uTorrent has to fill them with zeroes. That creates significant disk load and affects overall system performance. Windows XP provides a way to allocate files of any size almost instantly: SetFileValidData() function. It won't work without SE_MANAGE_VOLUME_NAME privilege (admin rights I guess) and with compressed files but I think it won't hurt anything in case of failure. Here is a link to small program that demonstrates this concept: http://depositfiles.com/files/8072561"

 

original (and more details) here:

http://forum.utorrent.com/viewtopic.php?pid=361352

 

I hope I could help you, so you can improve ImgBurn and so help me at the end.

Let me know if you need more info.

[LIGHTNING UK!]

I'd already seen / looked at that function actually

 

http://msdn.microsoft.com/en-us/library/aa365544(VS.85).aspx

 

... but says it can't be a network file.

[LIGHTNING UK!]

Do you have the sparse files option enabled in uTorrent? ('options -> preferences -> advanced -> diskio.sparse_files' I believe)

[kata23]

Sparse files are set to false in uTorrent.

 

I tried also some different things.

I can access a NSS volume on a Netware server via several ways, but the windows internal CIFS and the Netware Client native are the most common.

Here are the results:

1.

ImgBurn detects NSS filesystem as FAT:

CIFS: yes, Netware Client: No

2.

Using ImgBurn "Create image file from disk" command to create an image

CIFS: it takes 1 minute to create file, Netware Client: it takes less then 1 second

3.

Using uTorrent to open a 6GB torrent:

CIFS: it takes 2 minutes to create file, Netware Client: it takes less then 1 second

 

So there is no problem with the speed, this is how windows (CIFS) handles it, uTorrent is same fast as ImgBurn, sorry for not testing it throughly enough!

 

The only problem is that the windows api returns FAT for NSS volumes over CIFS, but if it can be overriden with the set file splitting to 'None' option, this problem is finished.

(Now I have to find that option only.)

[kata23]

Ok, meanwhile I found it: Settings / Read / Options / File Splitting.

[LIGHTNING UK!]

I believe the program will still double check and basically ignore that you've set it to 'None' - because a lot of users used to do that without really knowing / understanding the implications. You'll have to wait for 2.4.4.0 for the program to respect (without question) exactly what you put in the file splitting box (it's a hidden option though).

 

Both Read mode and Build mode have their own file splitting option btw.

 

I find it quite funny that even if you map a drive to a share (on a FAT32 drive) on the same box (WinXP SP3), it still only reports 'FAT' (rather than 'FAT32') for the file system. Luckily, mapping to an NTFS drive does still show NTFS as the FS on the mapped drive.