Watch, Follow, &
Connect with Us

For forums, blogs and more please visit our
Developer Tools Community.


ID: 17728, FileSortDemo

by Dmitry Surkov Email: Anonymous


The FileSortDemo implements sorting of the very large text files and serves as a teaching example of how visual feedback should be programmed in good multithreaded applications.
Download Details
FTP  download also available
CDN Login Required to Download. (You will be redirected to the login page if you click on the Download Link)
To download this, you must have registered:
A free membership

For Delphi, Version 6.0  to 6.0 791 downloads
Copyright: No significant restrictions


Size: 10,745 bytes
Updated on Tue, 26 Mar 2002 10:40:03 GMT
Originally uploaded on Tue, 26 Mar 2002 10:19:42 GMT
SHA1 Hash: CF222B565E1DE6826731A9111A8FFA9175FD6385
MD5 Hash: C9A9A95E21E691CA8D0905D5FC808D91

    Explore the files in this upload

Description
The FileSortDemo application implements sorting of the very large
text files using the FileSort algorithm (combination of the
QuickSort and MergeSort algorithms). The sorting is done in a
separate thread. At the time of sorting the application main thread
provides progress visualization and the means to interrupt the too
long processing.

The main idea behind the FileSortDemo example is to demonstrate
techniques, which in the opinion of the author should be used to
implement good multithreaded applications with threads visual
feedback. The main requirements for multithreaded applications
are:

1. Each worker thread should run as fast as possible. Visual
feedback should not affect thread performance considerably.
2. Application should remain responsive during all time of the
worker thread execution.
3. Application should provide means for the user to interrupt
too long processing, and to restart processing with the new
parameters.

Deep analysis of these essential requirements leads to the following
conclusions:

The worker thread should not be slowed down by the frequent
synchronization with the main thread in order to provide visual
feedback. The worker thread should not provide visual feedback
itself. Visualization task should be primarily performed by another
thread, e.g. by the main thread. Physically this means that the
worker thread should not call Synchronize method frequently for
updating the progress status.

The data exchange between worker thread and the main thread
should be based on the mechanism of the shared data with the
synchronized access via critical section. We will refer to these data
as mutable data. In the FileSortDemo example the mutable data are
progress data: progress position, progress limit, and text of the
progress label.

The mutable data is better to store in a thread object. The
TDataProcessingThread class (see DataProcThrd.pas unit) extends
the TThread class and implements the interface for writing data by
the thread itself and reading data by the main thread:

..TDataProcessingThread = class(TThread)
..private
....FDataSync: TCriticalSection;
....FDataChanged: Boolean;
..protected
....procedure BeginWrite; virtual; // Use inside thread
....procedure EndWrite; virtual; // Use inside thread
..public
....constructor Create(CreateSuspended: Boolean);
....destructor Destroy; override;
....procedure BeginRead; virtual; // Use outside thread
....procedure EndRead; virtual; // Use outside thread
....property DataChanged: Boolean read FDataChanged;
..end;

The scenario of using TDataProcessingThread methods and
properties is following. When worker thread needs to update
progress, it calls BeginWrite, writes new progress values, and then
calls EndWrite. The main thread periodically (on timer events)
checks the DataChanged property. If it's True, then the main thread
calls BeginRead method, reads new progress values, and then calls
EndRead.

Implementation of the BeginWrite, EndWrite, BeginRead,
EndRead methods is trivial:

procedure TDataProcessingThread.BeginWrite;
begin
..FDataSync.Enter;
end;

procedure TDataProcessingThread.EndWrite;
begin
..FDataChanged := True;
..FDataSync.Leave;
end;

procedure TDataProcessingThread.BeginRead;
begin
..FDataSync.Enter;
end;

procedure TDataProcessingThread.EndRead;
begin
..FDataChanged := False;
..FDataSync.Leave;
end;

Note, that the try...finally...end block is used to work with
BeginWrite, EndWrite, BeginRead, EndRead methods and protect
manipulations with mutable data from exceptions. This block
guaranties that the critical section will be released in case of
exception. The exception raised between calls to
BeginWrite/BeginRead and EndWrite/EndRead will not cause
deadlocks (we recommend to use try...finally...end block even if
between calls to BeginWrite/BeginRead and EndWrite/EndRead
you simply assign one variable to another).

The TFileSortThread (see FileSortThrd.pas unit) extends the
TDataProcessingThread and implements the FileSort algorithm.

Important possibility provided by the BeginRead and EndRead
methods is to indirectly suspend and resume thread execution in a
stable state, when the thread data can be accessed safely. This may
be a good technique when you want to move your multithreaded
application to Kylix and Linux platform, where Suspend/Resume
may not work properly (due to other threading model).

Another important issue is control over the thread lifetime. In the
FileSortDemo application thread is responsible to free itself. The
thread lifetime form the main thread point of view is accessibility
of the FSortThread field (see DemoForm form in MainUnit.pas
unit). FSortThread field stores reference to the TFileSortThread
object. If FSortThread gets the nil value, then thread is considered
terminated.

When the user starts processing, the StartActionExecute procedure
is called. It creates suspended thread, sets up properties, and then
resumes thread.

procedure TDemoForm.StartActionExecute(Sender: TObject);
begin
..DisableControls;
..FSortThread := TFileSortThread.Create(
....InputFileEdit.Text, OutputFileEdit.Text, TempDirEdit.Text);
..FSortThread.BufferSize := BufferSizeUpDown.Position * 1024 * 1024;
..FSortThread.FreeOnTerminate := True;
..FSortThread.OnTerminate := ThreadTerminated;
..FSortThread.OnSynchronizeProgress := ThreadTimerTimer;
..FSortThread.OnCompare := CompareStrings;
..FSortThread.Resume;
..ThreadTimer.Enabled := True;
end;

Note, that when FreeOnTerminate property is set to True, the
OnTerminate event handler should be set in order to be notified
that the thread variable is no more valid.

When processing ends or terminates, the ThreadTerminated
procedure is called:

procedure TDemoForm.ThreadTerminated(Sender: TObject);
var
..Thread: TThread;
begin
..FSortThread := nil; // FSortThread will be inaccessible
..Thread := Sender as TThread; // Sender is a terminating thread
..ThreadTimer.Enabled := False;
..if Assigned(Thread.FatalException) then
....Application.ShowException(Exception(Thread.FatalException));
..EnableControls;
end;

Note that the FatalException property is checked and the error
message is displayed. The OnTerminate event handler is the only
place, where you can do it safely for thread, which frees itself (i.e. when FreeOnTerminate = True).

The Timer component is used to update thread progress. The timer
event handler looks like following:

procedure TDemoForm.ThreadTimerTimer(Sender: TObject);
begin
..if Assigned(FSortThread) then
....if FSortThread.DataChanged then
....begin
......FSortThread.BeginRead;
......try
........ProgressLabel.Caption := FSortThread.Text;
........if FSortThread.Total <> 0 then
..........ProgressBar.Position := FSortThread.Position *
............ProgressBar.Max div FSortThread.Total
........else
..........ProgressBar.Position := 0;
......finally
........FSortThread.EndRead;
......end;
....end;
end;

Note, that the try...finally...end block is used to work with
BeginRead and EndRead methods.

When the user stops the thread execution, the FormCloseQuery
procedure is called. It uses BeginRead/EndRead to suspend/resume
thread execution and to ask user for confirmation:

procedure TDemoForm.FormCloseQuery(Sender: TObject;
..var CanClose: Boolean);
begin
..CanClose := not Assigned(FSortThread);
..if Assigned(FSortThread) and CloseAction.Enabled then
..begin
....FSortThread.BeginRead; // instead of FSortThread.Suspend
....try
......if MessageDlg('Do you want to stop sorting of file?',
........mtConfirmation, [mbYes, mbNo], 0) = mrYes then
........// Thread could finish, so check that FSortThread <> nil
........if Assigned(FSortThread) then
........begin
..........CloseAction.Enabled := False;
..........ProgressLabel.Caption := 'Cancelling...';
..........FSortThread.Terminate;
........end;
....finally
......if Assigned(FSortThread) then
........FSortThread.EndRead; // instead of FSortThread.Resume
....end;
..end;
end;

This description provides no information about implementation of
the FileSort algorithm. Implementation is not very optimal, but it's
simple enough to be understandable without comments.

Copyright: Dmitry Surkov.

Contest entry forMultithreaded programming

   Latest Comments  View All Add New

Move mouse over comment to see the full text

Could not retrieve comments. Please try again later.

Server Response from: ETNACDC03