Discussion:
lack of speed in BCB 2006
(too old to reply)
Thomas
2008-06-11 04:11:25 UTC
Permalink
hello
i have a program that is written by Borland C++Builder 6
it is a program for reading/writing binary files using ifstream and ofstream.
but when i ship this program to Borland C++Builder 2006 and compile it,the run time of program increased very long.
you can test ifstream/ofstream speed in both BCB 6 and BCB 2006
the time of BCB 6 is very very better than BCB 2006
can you help me?
Darko Miletic
2008-06-11 13:43:47 UTC
Permalink
Post by Thomas
hello
i have a program that is written by Borland C++Builder 6
it is a program for reading/writing binary files using ifstream and ofstream.
but when i ship this program to Borland C++Builder 2006 and compile it,the run time of program increased very long.
you can test ifstream/ofstream speed in both BCB 6 and BCB 2006
the time of BCB 6 is very very better than BCB 2006
can you help me?
Show us your sample code. In general it is not good idea to use IOstream
library for binary files.
Thomas
2008-06-11 14:11:33 UTC
Permalink
Post by Darko Miletic
Post by Thomas
hello
i have a program that is written by Borland C++Builder 6
it is a program for reading/writing binary files using ifstream and ofstream.
but when i ship this program to Borland C++Builder 2006 and compile it,the run time of program increased very long.
you can test ifstream/ofstream speed in both BCB 6 and BCB 2006
the time of BCB 6 is very very better than BCB 2006
can you help me?
Show us your sample code. In general it is not good idea to use IOstream
library for binary files.
what is your idea for working with binary files?
Darko Miletic
2008-06-11 14:27:32 UTC
Permalink
Post by Thomas
Post by Darko Miletic
Show us your sample code. In general it is not good idea to use IOstream
library for binary files.
what is your idea for working with binary files?
Underneath IOstreams use C file functions (fopen, fread, fwrite etc.)
and they introduce additional slowdown. That is why I do not recommend
working with binary files using IOstream library.

However if you really need that it is recomended to open file as binary
and use only read and write methods of fstream class. Something like this:

std::fstream fs("somefile",std::ios_base::binary);
//this is needed to guarantee that no data will be changed in any way.
fs.imbue(std::locale("C"));

//for writing
fs.write(mybuffer,mybuffersize);

//for reading
fs.read(inmybuffer,maxReadSize);


Even with all that it will still probably be slower than you want. You
should consider using memory mapped files and C file functions which
would give you maximum speed.

There is also another (faster) alternative to IOstreams called FASTreams
(http://www.msobczak.com/prog/fastreams/)
Darko Miletic
2008-06-11 15:09:22 UTC
Permalink
Post by Darko Miletic
There is also another (faster) alternative to IOstreams called FASTreams
(http://www.msobczak.com/prog/fastreams/)
I should just note that FASTreams do not work with BDS 2006 which is
rather unfortunate.
Thomas
2008-06-11 15:18:08 UTC
Permalink
Post by Darko Miletic
Post by Thomas
Post by Darko Miletic
Show us your sample code. In general it is not good idea to use IOstream
library for binary files.
what is your idea for working with binary files?
Underneath IOstreams use C file functions (fopen, fread, fwrite etc.)
and they introduce additional slowdown. That is why I do not recommend
working with binary files using IOstream library.
However if you really need that it is recomended to open file as binary
std::fstream fs("somefile",std::ios_base::binary);
//this is needed to guarantee that no data will be changed in any way.
fs.imbue(std::locale("C"));
//for writing
fs.write(mybuffer,mybuffersize);
//for reading
fs.read(inmybuffer,maxReadSize);
Even with all that it will still probably be slower than you want. You
should consider using memory mapped files and C file functions which
would give you maximum speed.
There is also another (faster) alternative to IOstreams called FASTreams
(http://www.msobczak.com/prog/fastreams/)
thanks Darko
but i have problem even with simple cases
for example you can test this simple code(or anything like this) in BCB6 and BCB2006 to see what i say:

#include <fstream>
#include <iostream>
#include <DateUtils.hpp>


void __fastcall TForm1::Button1Click(TObject *Sender)
{
int n[100000];

for(int i=0; i<100000; i++)
n[i] = 0;

ofstream out("test", ios::out | ios::binary);
if(!out) {
ShowMessage("Cannot open file.");
}

TTime t1=Time ();

out.write((char *) &n, sizeof n);

TTime t2=Time ();

Edit1->Text= IntToStr(MilliSecondsBetween(t1,t2));

out.close();
}
Darko Miletic
2008-06-11 15:36:13 UTC
Permalink
Post by Thomas
thanks Darko
but i have problem even with simple cases
ces't la vie. New Standard c++ library shipped with BDS is slower since
it depends on heavy compiler inlining which is something BDS does not do
so well.

You will have to stop using IOStreams if you need speed.

Use memory mapped files and C file functions.
Thomas
2008-06-11 18:33:10 UTC
Permalink
Post by Darko Miletic
Post by Thomas
thanks Darko
but i have problem even with simple cases
ces't la vie. New Standard c++ library shipped with BDS is slower since
it depends on heavy compiler inlining which is something BDS does not do
so well.
You will have to stop using IOStreams if you need speed.
Use memory mapped files and C file functions.
i have used some other methods
but all of them aren't as good as fstream in both compilers

this is my resulted table :
(times are MilliSecond)


|fstream |TFileStream |File |TMemoryStream
--------------------------------------------------------------
BCB 2006 time |344-390 | 124-181 |104-188 |233-250
BCB 6 time |0 - 16 | 125-150 |104-125 |234-280


do you have any idea?
Darko Miletic
2008-06-11 20:20:32 UTC
Permalink
Post by Thomas
(times are MilliSecond)
|fstream |TFileStream |File |TMemoryStream
--------------------------------------------------------------
BCB 2006 time |344-390 | 124-181 |104-188 |233-250
BCB 6 time |0 - 16 | 125-150 |104-125 |234-280
do you have any idea?
I think your example is misleading here is modified example that uses
only STL and windows API:

//---------------------------------------------------------------------------
#include <iostream>
#include <fstream>
#include <locale>
#include <windows.h>
//---------------------------------------------------------------------------

void dotest() {
int n[10000];
DWORD dwStart, dwEnd, dwTotal = 0;
std::ofstream out;
for (int pos = 0; pos < 10000; pos++) {
n[pos] = 0;
out.open("test",std::ios::binary);
if (!out.is_open()) {
std::cout << "Error" << std::endl;
}
out.imbue(std::locale::classic());
dwStart = ::GetTickCount();
out.write((char*)&n,sizeof(n));
dwEnd = ::GetTickCount();
dwTotal += (dwEnd - dwStart);
out.close();
out.clear();
}
std::cout << dwTotal << " msecs" << std::endl;
}

HANDLE openfile(const char* fname) {
HANDLE fileHandle_ = ::CreateFile( fname,
GENERIC_WRITE,
0,
NULL,
CREATE_NEW,
FILE_ATTRIBUTE_NORMAL,
NULL );
if (fileHandle_ == INVALID_HANDLE_VALUE)
fileHandle_ = NULL;
return fileHandle_;
}

void dotest2() {
int n[10000];
DWORD dwStart, dwEnd, dwTotal = 0;
DeleteFile("test");
HANDLE out=NULL;
DWORD dwrt = 0;
for (int pos = 0; pos < 10000; pos++) {
n[pos] = 0;
out = openfile("test");
if (NULL == out) {
std::cout << "Error" << std::endl;
break;
}
dwStart = ::GetTickCount();
WriteFile(out,&n[0],sizeof(n),&dwrt,NULL);
dwEnd = ::GetTickCount();
CloseHandle(out);
DeleteFile("test");
out = NULL;
dwTotal += (dwEnd - dwStart);
}
std::cout << dwTotal << " msecs" << std::endl;
}


int __cdecl main(int argc, char* argv[]) {
(argc), (argv);

dotest();
dotest2();
return 0;
}
//---------------------------------------------------------------------------
Both executables compiled from command line like this:
bcc32 -tWC -v- -k- -O2 -w Unit1.cpp

Here are my results:
time total (STL) time total(API)
Turbo C++ (BDS 2006) : 37463 msecs 1547 msecs
Borland Free compiler (5.5.1): 1142 msecs 1421 msecs

Accept that BDS has slower STL and RTL. Instead stop using IOStream or
VCL and turn to Windows API which is the faster way.
Thomas
2008-06-12 11:45:24 UTC
Permalink
Post by Darko Miletic
time total (STL) time total(API)
Turbo C++ (BDS 2006) : 37463 msecs 1547 msecs
Borland Free compiler (5.5.1): 1142 msecs 1421 msecs
Accept that BDS has slower STL and RTL. Instead stop using IOStream or
VCL and turn to Windows API which is the faster way.
ok, thanks Darko
Using Windows API is faster (in both compilers)
but only in Reading the file!
on Writing i still have same problem (in both compilers)
Darko Miletic
2008-06-12 13:05:19 UTC
Permalink
Post by Thomas
Post by Darko Miletic
time total (STL) time total(API)
Turbo C++ (BDS 2006) : 37463 msecs 1547 msecs
Borland Free compiler (5.5.1): 1142 msecs 1421 msecs
Accept that BDS has slower STL and RTL. Instead stop using IOStream or
VCL and turn to Windows API which is the faster way.
ok, thanks Darko
Using Windows API is faster (in both compilers)
but only in Reading the file!
on Writing i still have same problem (in both compilers)
What do you mean faster only in reading? The code I sent is for WRITING
and it is faster than standard code in BDS. What it is that you are
measuring?
Thomas
2008-06-13 13:38:59 UTC
Permalink
Post by Darko Miletic
What do you mean faster only in reading? The code I sent is for WRITING
and it is faster than standard code in BDS. What it is that you are
measuring?
please test this source and see the resulted time :
(not difference what compiler)

#include <DateUtils.hpp>

HANDLE OpenWrite(const char* fname) {
HANDLE fileHandle = CreateFile( fname,
GENERIC_WRITE,
0,
NULL,
CREATE_ALWAYS,
FILE_ATTRIBUTE_NORMAL,
NULL );

if (fileHandle == INVALID_HANDLE_VALUE)
fileHandle = NULL;
return fileHandle;
}


HANDLE OpenRead(const char* fname) {
HANDLE fileHandle = CreateFile( fname,
GENERIC_READ,
0,
NULL,
OPEN_EXISTING,
FILE_ATTRIBUTE_NORMAL,
NULL );
if (fileHandle == INVALID_HANDLE_VALUE)
fileHandle = NULL;
return fileHandle;
}


void writefile(){
int n[100000];
DWORD dwrt = 0;
HANDLE out=NULL;

out = OpenWrite("test");
if (out == NULL){
ShowMessage("Cannot open file.");
exit(0);
}

for (int pos = 0; pos < 100000; pos++) {
n[pos] = 1;
}

TTime t1=Time ();
WriteFile(out,n,sizeof(n),&dwrt,NULL);
TTime t2=Time ();
ShowMessage("time of writing : "+ IntToStr(MilliSecondsBetween(t1,t2)));

CloseHandle(out);
out = NULL;
}


void readfile(){
int n[100000];
HANDLE in=NULL;
DWORD dwrt = 0;

in = OpenRead("test");
if (in == NULL){
ShowMessage("Cannot open file.");
exit(0);
}

TTime t1=Time ();
ReadFile(in,n,sizeof(n),&dwrt,NULL);
TTime t2=Time ();
ShowMessage("time of reading : "+ IntToStr(MilliSecondsBetween(t1,t2)));

CloseHandle(in);
in = NULL;
}
Darko Miletic
2008-06-13 14:08:00 UTC
Permalink
Post by Thomas
Post by Darko Miletic
What do you mean faster only in reading? The code I sent is for WRITING
and it is faster than standard code in BDS. What it is that you are
measuring?
(not difference what compiler)
As I said in my previous message read through link's and experiment with
options related to CreateFile API and memory mapped files. It is not up
to me to do your job of investigation :) I'm just pointing you to the
desired direction.
Chris Uzdavinis (TeamB)
2008-06-13 17:30:52 UTC
Permalink
Post by Thomas
HANDLE OpenWrite(const char* fname) {
HANDLE fileHandle = CreateFile( fname,
GENERIC_WRITE,
0,
NULL,
CREATE_ALWAYS,
FILE_ATTRIBUTE_NORMAL,
NULL );
if (fileHandle == INVALID_HANDLE_VALUE)
fileHandle = NULL;
return fileHandle;
}
Something just stands out as "wrong" in the above code, in the
assignment of NULL to fileHandle. If INVALID_HANDLE_VALUE is the same
value as NULL, then the assignment is unnecessary. If the values are
different, then NULL could overlap a legitimate value returned by
CreateFile, in which case the caller would have no way to distinguish
whether it's valid or not.

What's wrong with indicating an invalid handle by using
INVALID_HANDLE_VALUE, and just doing this:


HANDLE OpenWrite(const char* fname) {
return CreateFile( fname,
GENERIC_WRITE,
0,
NULL,
CREATE_ALWAYS,
FILE_ATTRIBUTE_NORMAL,
NULL );
}


I haven't considered the arguments passed to CreateFile because I
haven't memorized them. But the caller should check the result
against the standard invalid handle, then by using NULL as if it were
the same thing.
--
Chris (TeamB);
Darko Miletic
2008-06-13 18:52:29 UTC
Permalink
Post by Chris Uzdavinis (TeamB)
Post by Thomas
HANDLE OpenWrite(const char* fname) {
HANDLE fileHandle = CreateFile( fname,
GENERIC_WRITE,
0,
NULL,
CREATE_ALWAYS,
FILE_ATTRIBUTE_NORMAL,
NULL );
if (fileHandle == INVALID_HANDLE_VALUE)
fileHandle = NULL;
return fileHandle;
}
Something just stands out as "wrong" in the above code, in the
assignment of NULL to fileHandle. If INVALID_HANDLE_VALUE is the same
value as NULL, then the assignment is unnecessary. If the values are
different, then NULL could overlap a legitimate value returned by
CreateFile, in which case the caller would have no way to distinguish
whether it's valid or not.
INVALID_HANDLE_VALUE is not equal to NULL. It is -1. NULL could never
overlap the valid value since assignment will occur only if fileHandle
is equal to INVALID_HANDLE_VALUE.

It may be better to write this just to make it clearer:

if (fileHandle == INVALID_HANDLE_VALUE) {
fileHandle = NULL;
}
Post by Chris Uzdavinis (TeamB)
What's wrong with indicating an invalid handle by using
Question of taste I guess.
Bob Gonder
2008-06-13 19:21:32 UTC
Permalink
Post by Darko Miletic
INVALID_HANDLE_VALUE is not equal to NULL. It is -1. NULL could never
overlap the valid value since assignment will occur only if fileHandle
is equal to INVALID_HANDLE_VALUE.
Umm, yes.
Chris was worried that CreateFile might open a file,
assigned 0 to it, and the wrapper returns the 0 signifying
to the application (incorrectly) that the open failed.

An unlikely (but theoreticaly possible) event.
IIRC handle 0 is one of the C standard handles,
and not too likely to be created/returned by this API.

Bob Gonder
2008-06-13 19:13:45 UTC
Permalink
Post by Chris Uzdavinis (TeamB)
Something just stands out as "wrong" in the above code, in the
assignment of NULL to fileHandle. If INVALID_HANDLE_VALUE is the same
value as NULL, then the assignment is unnecessary. If the values are
different, then NULL could overlap a legitimate value returned by
CreateFile, in which case the caller would have no way to distinguish
whether it's valid or not.
In theory, you are correct.
In practice, it doesn't usualy matter.
INVALID_HANDLE_VALUE is -1.
A handle of 0 would be one of the std handles,
not a new file handle, so confusion is minimal.
Manipulating the return such that bool works
properly might be a good thing?

if( (handle=Open(file)) )
vs
if( INVALID_HANDLE_VALUE != (handle=Open(file)) )

I'd also be tempted to check GetLastError() within the wrapper.
Darko Miletic
2008-06-12 13:03:41 UTC
Permalink
Post by Thomas
ok, thanks Darko
Using Windows API is faster (in both compilers)
but only in Reading the file!
on Writing i still have same problem (in both compilers)
Explore documentation for CreateFile API. There are lot of options
regarding caching opened file during reading and/or writing etc. You
will just have to experiment.

CreateFile
http://msdn.microsoft.com/en-us/library/aa363858(VS.85).aspx

File Caching
http://msdn.microsoft.com/en-us/library/aa364218(VS.85).aspx

File Buffering
http://msdn.microsoft.com/en-us/library/cc644950(VS.85).aspx


Also take a look at memory mapped files because as stated in MSDN:
"The use of file mapping improves efficiency because the file resides on
disk, but the file view resides in memory."

File mapping
http://msdn.microsoft.com/en-us/library/aa366556(VS.85).aspx

CreateFileMapping
http://msdn.microsoft.com/en-us/library/aa366537.aspx

MapViewOfFile
http://msdn.microsoft.com/en-us/library/aa366761(VS.85).aspx

Reading and writing from file view
http://msdn.microsoft.com/en-us/library/aa366801(VS.85).aspx
dnettels
2008-06-13 14:43:10 UTC
Permalink
Post by Darko Miletic
ces't la vie. New Standard c++ library shipped with BDS is slower since
it depends on heavy compiler inlining which is something BDS does not do
so well.
You will have to stop using IOStreams if you need speed.
Use memory mapped files and C file functions.
When I read lines like this I get a bit frustrated about my choice of
using BDS. Why is the CodeGear compiler so much behind other compilers?
Or do other complilers (VC++, gcc...) also have such problems with
proper inlining or any other things that make the use of the STL
inefficient? It would be much more elegant if one could just use
iostream in binary mode (using read() and write()) without a speed
penalty of more than a factor 10. Am I wrong when I say that in general
efficient template programming depends heavily on appropriate inlining?

My hope is that the promised (and eagerly awaited) standard conformance
improvements of the coming bds will also address such problems.
I very much like the RAD facilities of the BDS. But at the end I also
want to be able to write and use (boost...) modern, efficient C++ code.

Daniel
Loading...