Watch, Follow, &
Connect with Us

For forums, blogs and more please visit our
Developer Tools Community.


Welcome, Guest
Guest Settings
Help

Thread: Using LockFileEx to wait for file becoming available



Permlink Replies: 5 - Last Post: Sep 27, 2016 11:50 PM Last Post By: Gerrit Beuze Threads: [ Previous | Next ]
Gerrit Beuze

Posts: 70
Registered: 10/16/00
Using LockFileEx to wait for file becoming available
Click to report abuse...   Click to reply to this thread Reply
  Posted: Sep 27, 2016 7:06 AM
Hi all,

When saving files to a directory that is sycned with the Google Drive app, I frequently run into
TFileStream.Create(fmCreate) raising an exception because the Google Drive app has the file locked while uploading
it to the server.

Save is done by writing a zip file (using ZipForge) that internally uses a
TFileStream in fmCreate mode to actually write the data.

I thought I could use LockFileEx to wait until the Google Drive app is ready and the file becomes available again like so:

var
FileHandle: THandle;
Overlapped: TOverlapped;
begin;
FileHandle := CreateFile(PChar(SourceFile), GENERIC_READ, FILE_SHARE_READ, nil, OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, 0);
if (FileHandle <> INVALID_HANDLE_VALUE) then
begin
try
ZeroMemory(@Overlapped, SizeOf(Overlapped));
if (LockFileEx(FileHandle, LOCKFILE_EXCLUSIVE_LOCK, 0, 0, MAXDWORD, Overlapped)) then
begin
try
// RAISES IOException HERE:
TArchiveWriter.CreateArchive(SourceFile, MindMapFileName, ImageHashes.List, FImageStore);
FileAge(SourceFile, FTimeStamp);
Exit(True);
finally
UnlockFileEx(FileHandle, 0, 0, MAXDWORD, Overlapped);
end;
end
else
Assert(OutputDebugStr('Unable to aquire lock'));
finally
CloseHandle(FileHandle);
end;
end
end;

The problem is however that when TArchiveWriter.CreateArchive does this:

FCompressedStream := TFileStream.Create(FileName, fmCreate)

TFileStream raises an exception about not being able to access the because the file is being
used by another process. Even after shutting down the Google Drive app. IOW: The aboev code
effective blocks TFileStream.Create

What am I doing wrong / not understanding correct:
It it my understanding that LockFile should grant access to the entire process, not just to that call

Thanks in advance,

Gerrit Beuze
ModelMaker Tools

Peter Below

Posts: 1,227
Registered: 12/16/99
Re: Using LockFileEx to wait for file becoming available
Click to report abuse...   Click to reply to this thread Reply
  Posted: Sep 27, 2016 9:20 AM   in response to: Gerrit Beuze in response to: Gerrit Beuze
Gerrit Beuze wrote:

Hi all,

When saving files to a directory that is sycned with the Google Drive
app, I frequently run into TFileStream.Create(fmCreate) raising an
exception because the Google Drive app has the file locked while
uploading it to the server.

Save is done by writing a zip file (using ZipForge) that internally
uses a TFileStream in fmCreate mode to actually write the data.

I thought I could use LockFileEx to wait until the Google Drive app
is ready and the file becomes available again like so:

var
FileHandle: THandle;
Overlapped: TOverlapped;
begin;
FileHandle := CreateFile(PChar(SourceFile), GENERIC_READ,
FILE_SHARE_READ, nil, OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, 0); if
(FileHandle <> INVALID_HANDLE_VALUE) then begin
try
ZeroMemory(@Overlapped, SizeOf(Overlapped));
if (LockFileEx(FileHandle, LOCKFILE_EXCLUSIVE_LOCK, 0, 0,
MAXDWORD, Overlapped)) then begin
try
// RAISES IOException HERE:
TArchiveWriter.CreateArchive(SourceFile, MindMapFileName,
ImageHashes.List, FImageStore); FileAge(SourceFile,
FTimeStamp); Exit(True);
finally
UnlockFileEx(FileHandle, 0, 0, MAXDWORD, Overlapped);
end;
end
else
Assert(OutputDebugStr('Unable to aquire lock'));
finally
CloseHandle(FileHandle);
end;
end
end;

The problem is however that when TArchiveWriter.CreateArchive does
this:

FCompressedStream := TFileStream.Create(FileName, fmCreate)

TFileStream raises an exception about not being able to access the
because the file is being used by another process.

Not by another, by your's! You get a lock on the open file but then
TArchiveWriter tries to open the file again. That is where it blows up.
The lock is per file handle, not per process.


--
Peter Below
TeamB

Gerrit Beuze

Posts: 70
Registered: 10/16/00
Re: Using LockFileEx to wait for file becoming available
Click to report abuse...   Click to reply to this thread Reply
  Posted: Sep 27, 2016 9:29 AM   in response to: Peter Below in response to: Peter Below
Hello Peter,

TFileStream raises an exception about not being able to access the
because the file is being used by another process.

Not by another, by your's! You get a lock on the open file but then
TArchiveWriter tries to open the file again. That is where it blows up.
The lock is per file handle, not per process.

I figured something like that.
I tried another approach, basically:

save .zip to temp file,

FileHandle := CreateFile(PChar(SourceFile), GENERIC_READ or GENERIC_WRITE, 0, nil, OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, 0);
if (FileHandle <> INVALID_HANDLE_VALUE) then
begin
ZeroMemory(@Overlapped, SizeOf(Overlapped));
Win32Check(LockFileEx(FileHandle, LOCKFILE_EXCLUSIVE_LOCK, 0, 0, MAXDWORD, Overlapped));

Copy temp file using 2 TFileStreams:
Target := TFileStream.Create(FileHandle);

etc

This works OK, except, that when Google Drive locks the file,
the call to
FileHandle := CreateFile( )

fails: with "unable to open file, file opened by another process."

It seems a chicken egg problem: I need to get a file handle to be able to lock it
but when it's locked I cannot get the file handle via CreateFile.

Is LockFileEx intended for situations like this?

What other approach could I use?

I thought about spinning
FileHandle := CreateFile()
until it succeeds, perhaps with a small wait() delay?

Gerrit Beuze
ModelMaker Tools
Peter Below

Posts: 1,227
Registered: 12/16/99
Re: Using LockFileEx to wait for file becoming available
Click to report abuse...   Click to reply to this thread Reply
  Posted: Sep 27, 2016 9:41 AM   in response to: Gerrit Beuze in response to: Gerrit Beuze
Gerrit Beuze wrote:

Hello Peter,

TFileStream raises an exception about not being able to access the
because the file is being used by another process.

Not by another, by your's! You get a lock on the open file but then
TArchiveWriter tries to open the file again. That is where it blows
up. The lock is per file handle, not per process.

I figured something like that.
I tried another approach, basically:

save .zip to temp file,

FileHandle := CreateFile(PChar(SourceFile), GENERIC_READ or
GENERIC_WRITE, 0, nil, OPEN_ALWAYS, FILE_ATTRIBUTE_NORMAL, 0); if
(FileHandle <> INVALID_HANDLE_VALUE) then begin
ZeroMemory(@Overlapped, SizeOf(Overlapped));
Win32Check(LockFileEx(FileHandle, LOCKFILE_EXCLUSIVE_LOCK, 0,
0, MAXDWORD, Overlapped));

Copy temp file using 2 TFileStreams:
Target := TFileStream.Create(FileHandle);

etc

This works OK, except, that when Google Drive locks the file,
the call to
FileHandle := CreateFile( )

fails: with "unable to open file, file opened by another process."

It seems a chicken egg problem: I need to get a file handle to be
able to lock it but when it's locked I cannot get the file handle via
CreateFile.

Is LockFileEx intended for situations like this?

No, it is intended to lock portions of a file in a scenario where
multiple writers may work on the same file.

What other approach could I use?

There is only one: try to open the file, (or just use MoveFileEx to
copy it), if it fails wait a bit and try again until you succeed. The
file system does not allow you to kind of reserve a file, it works on a
first come first served basis.

I thought about spinning
FileHandle := CreateFile()
until it succeeds, perhaps with a small wait() delay?

Yes, that is the only way. Try to create the file with exclusive access
until you succeed.


--
Peter Below
TeamB

Gerrit Beuze

Posts: 70
Registered: 10/16/00
Re: Using LockFileEx to wait for file becoming available
Click to report abuse...   Click to reply to this thread Reply
  Posted: Sep 27, 2016 11:50 PM   in response to: Peter Below in response to: Peter Below
Thanks Peter,

Gerrit Beuze

Is LockFileEx intended for situations like this?

No, it is intended to lock portions of a file in a scenario where
multiple writers may work on the same file.


What other approach could I use?

There is only one: try to open the file, (or just use MoveFileEx to
copy it), if it fails wait a bit and try again until you succeed. The
file system does not allow you to kind of reserve a file, it works on a
first come first served basis.

I thought about spinning
FileHandle := CreateFile()
until it succeeds, perhaps with a small wait() delay?

Yes, that is the only way. Try to create the file with exclusive access
until you succeed.
Remy Lebeau (Te...


Posts: 9,447
Registered: 12/23/01
Re: Using LockFileEx to wait for file becoming available
Click to report abuse...   Click to reply to this thread Reply
  Posted: Sep 27, 2016 10:58 AM   in response to: Gerrit Beuze in response to: Gerrit Beuze
Gerrit wrote:

When saving files to a directory that is sycned with the Google Drive
app, I frequently run into TFileStream.Create(fmCreate) raising an
exception because the Google Drive app has the file locked while
uploading it to the server.

If you are creating a new file, you would be creating it with exclusive access
rights by default (fmCreate by itself uses exclusive access), so Google Drive
should not be able to access the file until after you have finished saving
it and have closed it.

What you describe would make more sense if you are trying to overwrite an
existing file that is still in the process of being uploaded. In which case
Google is likely to have exclusive access (or at least deny writing access)
to the file so it does not get corrupted. You would have to wait for the
upload to finish before you can then overwrite the file.

I thought I could use LockFileEx to wait until the Google Drive app is
ready and the file becomes available again like so:

What you have shown will not work. You are opening the target file for read-only
access, obtaining an exclusive lock to it (irrelevant to your error), and
then trying to re-open the same target file for writing, which fails (asking
for write access to a file that is open without sharing write access).

You need to remove the read-only CreateFile() call. And LockFileEx() will
not solve your problem, either. Besides, if you do use CreateFile() manually,
you can ask it to obtain exclusive access (set the dwShareMode parameter
to 0), you don't need LockFileEx() for that.

Basically, don't open the file manually with CreateFile() at all, let CreateArchive()
create the file for you. If it fails, let it fail, since writing to the
file is not allowed at that moment (likely for good reason), try creating
the file again at a later time. You can't wait on it directly without being
able to open it in the first place.

The problem is however that when TArchiveWriter.CreateArchive
does this:

FCompressedStream := TFileStream.Create(FileName, fmCreate)

TFileStream raises an exception about not being able to access the
because the file is being used by another process.

Yes, because YOU have the file open, from your CreateFile() call.

Even after shutting down the Google Drive app.

Because YOU still have the file open.

It it my understanding that LockFile should grant access to the entire
process, not just to that call

That is not what LockFileEx() is meant for. You are misunderstanding and
misusing it.

--
Remy Lebeau (TeamB)
Legend
Helpful Answer (5 pts)
Correct Answer (10 pts)

Server Response from: ETNAJIVE02