wc3campaigns
WC3C Homepage - www.wc3c.netUser Control Panel (Requires Log-In)Engage in discussions with other users and join contests in the WC3C forums!Read one of our many tutorials, ranging in difficulty from beginner to advanced!Show off your artistic talents in the WC3C Gallery!Download quality models, textures, spells (vJASS/JASS), systems, and scripts!Download maps that have passed through our rigorous approval process!

Go Back   Wc3C.net > Warcraft III Modding > Developer's Corner > Warcraft Editing Tools > Programming
User Name
Password
Register Rules Get Hosted! Chat Pastebin FAQ and Rules Members List Calendar



Closed Thread
 
Thread Tools Search this Thread
Old 10-25-2004, 11:55 PM   #1
BlacKDicK
User


Respected User
 
Join Date: Mar 2002
Posts: 589

BlacKDicK is on a distinguished road (24)

Default

This dll is based on SFmpq static 1.0.7.4 and has some speed
improvements. I changed it´s version to 1.0.7.6, so you can
distinguish them by checking the version.
Everything else works the same as the original 1.0.7.4.
USAGE: Just replace your SFmpq.dll with the one included with
this package.

1.0.7.6 Fixed some bugs. Also, since WC3Campaigns is now back on business
I´m uploading it again. I had to compile this thing again, coz I didn´t found any binary
working version on my PC. Here are my BenchMark results against the latest Game.dll (1.20b)
Athlon XP 2000+ 1658 MHz (12,5 * 133) L1=64KB L2=256KB
RAM = 2x 512 MB @ 2,5/3/4/7/2
HD: PATA Maxtor 120 GB 7200 rpm 2MB cache

Code:
MpqBench 1.01 by BlacKDicK using SFmpq (1.0.7.6)
Benchmark File: C:\Jogos\Warcra~1\Game.dll
*****************************************
Adding buffer (9375804 bytes) to MPQ using (0x1), level (0)...
Time to add: 231.924169 msec
Reading file buffer from MPQ...
Time to read: 47.164374 msec
Testing read buffer...(MATCH)
*****************************************
Adding buffer (9375804 bytes) to MPQ using (0x2), level (9)...
Time to add: 1598.574959 msec
Reading file buffer from MPQ...
Time to read: 186.772062 msec
Testing read buffer...(MATCH)
*****************************************
Adding buffer (9375804 bytes) to MPQ using (0x8), level (0)...
Time to add: 1754.626636 msec
Reading file buffer from MPQ...
Time to read: 273.822511 msec
Testing read buffer...(MATCH)

And here is the original 1.0.7.4 results. As you can see, they look similar, except from the zlib-deflate compression, wich got pretty good improvements.
Code:
MpqBench 1.01 by BlacKDicK using SFmpq (1.0.7.4)
Benchmark File: C:\Jogos\Warcra~1\Game.dll
*****************************************
Adding buffer (9375804 bytes) to MPQ using (0x1), level (0)...
Time to add: 233.851509 msec
Reading file buffer from MPQ...
Time to read: 47.618901 msec
Testing read buffer...(MATCH)
*****************************************
Adding buffer (9375804 bytes) to MPQ using (0x2), level (9)...
Time to add: 4295.041409 msec
Reading file buffer from MPQ...
Time to read: 434.937960 msec
Testing read buffer...(MATCH)
*****************************************
Adding buffer (9375804 bytes) to MPQ using (0x8), level (0)...
Time to add: 1707.225639 msec
Reading file buffer from MPQ...
Time to read: 375.900289 msec
Testing read buffer...(MATCH)
Attached Files
File Type: rar SFmpq_fast.rar (112.1 KB, 200 views)

Last edited by BlacKDicK : 12-29-2005 at 01:24 AM. Reason: Uploading new version
BlacKDicK is offline  
Sponsored Links - Login to hide this ad!
Old 10-26-2004, 04:39 PM   #2
Vexorian
Free Software Terrorist
 
Vexorian's Avatar


Technical Director
 
Join Date: Apr 2003
Posts: 14,898

Submissions (37)

Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)

Hero Contest #3 - 2nd Place

Default

It sounds great, and it is also great you are back to this stuff, anyways I think the programming forum is a better place for it
__________________
Zoom (requires log in)Wc3 map optimizer 5.0
Someone should fix .wav sound in this thing.
Zoom (requires log in)JassHelper 0.A.2.A
Turns your simple code into something that is complicated enough to work.
Faster != more useful
Vexorian is offline  
Old 10-28-2004, 08:59 PM   #3
Vexorian
Free Software Terrorist
 
Vexorian's Avatar


Technical Director
 
Join Date: Apr 2003
Posts: 14,898

Submissions (37)

Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)

Hero Contest #3 - 2nd Place

Default

I think it is amd specific, I tested it in my pentiu 4 , and rebuilding with Zlib was leaving an empty mpq
__________________
Zoom (requires log in)Wc3 map optimizer 5.0
Someone should fix .wav sound in this thing.
Zoom (requires log in)JassHelper 0.A.2.A
Turns your simple code into something that is complicated enough to work.
Faster != more useful
Vexorian is offline  
Old 10-28-2004, 09:05 PM   #4
BlacKDicK
User


Respected User
 
Join Date: Mar 2002
Posts: 589

BlacKDicK is on a distinguished road (24)

Default

Quote:
Originally Posted by Lord Vexorian
I think it is amd specific, I tested it in my pentiu 4 , and rebuilding with Zlib was leaving an empty mpq
Thanks for the info. I´ll then develop separate versions. Hold on a bit and a P4 version should be coming along.
BlacKDicK is offline  
Old 03-14-2005, 01:49 AM   #5
BlacKDicK
User


Respected User
 
Join Date: Mar 2002
Posts: 589

BlacKDicK is on a distinguished road (24)

Default

I think i fixed the errors. Here is a new beta version.
NOTES: I need someone with an INTEL cpu (P4) to test this library for me.
It is using a "fast" memcpy() routine wich may have some problems on
INTEL based CPUs. On my AMD Athlon XP it is working nice.
Please read the included "ReadMe.txt" and post some results. BTW, it is about 3 times faster than the original version. :o

EDIT:
[March,19,2005] New version: Fixed some bugs, reduced library size a bit, changed version to 1.0.7.5 to avoid misunderstandings, MpqBench now tests 3 compression types.

Last edited by BlacKDicK : 03-19-2005 at 01:53 PM.
BlacKDicK is offline  
Old 03-14-2005, 09:02 PM   #6
Vexorian
Free Software Terrorist
 
Vexorian's Avatar


Technical Director
 
Join Date: Apr 2003
Posts: 14,898

Submissions (37)

Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)

Hero Contest #3 - 2nd Place

Default

I will go home and test it
__________________
Zoom (requires log in)Wc3 map optimizer 5.0
Someone should fix .wav sound in this thing.
Zoom (requires log in)JassHelper 0.A.2.A
Turns your simple code into something that is complicated enough to work.
Faster != more useful
Vexorian is offline  
Old 03-15-2005, 01:18 AM   #7
BlacKDicK
User


Respected User
 
Join Date: Mar 2002
Posts: 589

BlacKDicK is on a distinguished road (24)

Default

BTW, it seems I fixed the Zlib compression but messed up with the other compressions, damm I'm stupid lol.

Trying to locate that, I will probably have it fixed by tomorrow. Meanwhile, I´d appreciate some 1.18 Game.dll benchs.
BlacKDicK is offline  
Old 03-19-2005, 01:43 AM   #8
Starcraftfreak
PR & UMSWE Developer
 
Starcraftfreak's Avatar


Respected User
 
Join Date: Aug 2002
Posts: 1,642

Starcraftfreak will become famous soon enough (66)Starcraftfreak will become famous soon enough (66)Starcraftfreak will become famous soon enough (66)

Send a message via ICQ to Starcraftfreak Send a message via AIM to Starcraftfreak Send a message via MSN to Starcraftfreak
Default

I tested the benchmark today and I have to say that I get virtually the same results for both versions of SFMPQ. They were always in the same range for both versions of the library. I don't know whats wrong here.

Quote:
MpqBench 1.00 by BlacKDicK
Adding buffer to MPQ using Deflate + best speed...
Time to add: 1091.416926 msec
Reading file buffer from MPQ...
Time to read: 138.767916 msec
Testing read buffer...(MATCH)

System:
Athlon64 3000+ (754)
1024MB RAM (DDR400 Single-Channel)
Starcraftfreak is offline  
Old 03-19-2005, 01:55 PM   #9
BlacKDicK
User


Respected User
 
Join Date: Mar 2002
Posts: 589

BlacKDicK is on a distinguished road (24)

Default

Uploaded new version, fixed some things, improved MpqBench. See my post above for download.

Starcraftfreak, would you mind running the test again, with this new version?
BlacKDicK is offline  
Old 03-19-2005, 02:58 PM   #10
Starcraftfreak
PR & UMSWE Developer
 
Starcraftfreak's Avatar


Respected User
 
Join Date: Aug 2002
Posts: 1,642

Starcraftfreak will become famous soon enough (66)Starcraftfreak will become famous soon enough (66)Starcraftfreak will become famous soon enough (66)

Send a message via ICQ to Starcraftfreak Send a message via AIM to Starcraftfreak Send a message via MSN to Starcraftfreak
Default

Optimized is slower (~ 100ms in writing, 10ms in reading for the last test). I don't want to post the results now, since I'm running some other programs as well, which makes them worse. I made another run with the older version after posting the results and I scored like 900/126 after I closed all programs.

But what is wrong with that optimization? Could it be that the Athlon64 is not suited for those?
Maybe you should also try SSE/SSE2.

Edit:
I found out what was wrong with the "regular" sfmpq.dll. It wasn't the regular one. It was the one of your WinMPQ modification which supports buffer size modification for new files. It was dated somewhere 2004. It contains the string "Modified by BlacKDicK based on SFmpq_static version 1,0,7,4" in the file comments.
Oddly, when I wanted to create a new UMSWE build to fix a little bug in the current one, it would always corrupt the listfile. I double checked and messed around a lot until I came up with the conclusion that something has to be wrong. Then I overwrote my SFmpq.dll with the original one from the WinMPQ download. The result was that it wrote a correct listfile into the MPQ but was considerably slower. Then I knew why this dll was not faster than the one "from WinMPQ". I also wonder how this problem occured. The last version of UMSWE was built with it and just everything is fine. I was using this dll for 4 months now and I never experienced any problems until today. Now I put your latest optimized version into my WinMPQ directory and it works just fine

I wonder if you ever released a similiarly optimized version some time ago (must be back in fall 2004). Today I had problems building a new version of UMSWE with the script which builds the MPQ with WinMPQ for me. Oddly, I created a new UMSWE build last week and it works fine.

I'll probably post new benchmarks a bit later.

Last edited by Starcraftfreak : 03-19-2005 at 10:29 PM.
Starcraftfreak is offline  
Old 03-24-2005, 09:44 PM   #11
BlacKDicK
User


Respected User
 
Join Date: Mar 2002
Posts: 589

BlacKDicK is on a distinguished road (24)

Default

Quote:
I wonder if you ever released a similiarly optimized version some time ago (must be back in fall 2004).
Yes I did, and I posted it here in this same topic. That version had some bugs when using the PkWare compression. Probably you only faced the bug when you first added files using this type of compression. Maybe the (listfile) worked before coz it got compressed with Zlib but now (dunno why) it was getting compressed with pkware.


This library can´t be slower than the original SFmpq. There are 3 separate things wich will help increase speed:
1) It is using zlib 1.22 , wich is faster.
2) I compiled the zlib using a special "Asm Version" wich can be found on the zlib package.
3) It is using the "fast" memcpy routine wich I got on the AMD dev site.

Even If we supose that this fast memcpy is not getting good improvements at all, there are still 2 other things helping the speed. Also, this library can have problems with Intel CPUs (I am not sure if fast memcpy is also using specific AMD code or just SSE) but on your K8 it should work flawlessly. I am pretty sure it ain´t using SSE2 coz my CPU (Athlon XP) does not support SSE2 and the library works just fine.

Btw, can you do a MpqBench using this new version and the regular SFmpq? I am curious about K8 performance coz I´ll probably buy one for me on july.

Thanks for the info. It seems that this version is working nice. If you find anything new, please let me know.

Last edited by BlacKDicK : 03-24-2005 at 09:50 PM.
BlacKDicK is offline  
Old 03-25-2005, 12:57 AM   #12
Starcraftfreak
PR & UMSWE Developer
 
Starcraftfreak's Avatar


Respected User
 
Join Date: Aug 2002
Posts: 1,642

Starcraftfreak will become famous soon enough (66)Starcraftfreak will become famous soon enough (66)Starcraftfreak will become famous soon enough (66)

Send a message via ICQ to Starcraftfreak Send a message via AIM to Starcraftfreak Send a message via MSN to Starcraftfreak
Default

Here are my results (I actually did those some days ago):
Quote:
MpqBench 1.01 by BlacKDicK using SFmpq (1.0.8.1)
Benchmark File: game.dll
*****************************************
Adding buffer (9371708 bytes) to MPQ using (0x1), level (0)...
Time to add: 124.444308 msec
Reading file buffer from MPQ...
Time to read: 29.469388 msec
Testing read buffer...(MATCH)
*****************************************
Adding buffer (9371708 bytes) to MPQ using (0x2), level (9)...
Time to add: 4834.831801 msec
Reading file buffer from MPQ...
Time to read: 687.857535 msec
Testing read buffer...(MATCH)
*****************************************
Adding buffer (9371708 bytes) to MPQ using (0x8), level (0)...
Time to add: 2647.195384 msec
Reading file buffer from MPQ...
Time to read: 565.053380 msec
Testing read buffer...(MATCH)

MpqBench 1.01 by BlacKDicK using SFmpq (1.0.7.5)
Benchmark File: game.dll
*****************************************
Adding buffer (9371708 bytes) to MPQ using (0x1), level (0)...
Time to add: 105.797804 msec
Reading file buffer from MPQ...
Time to read: 28.973515 msec
Testing read buffer...(MATCH)
*****************************************
Adding buffer (9371708 bytes) to MPQ using (0x2), level (9)...
Time to add: 1840.799878 msec
Reading file buffer from MPQ...
Time to read: 269.952187 msec
Testing read buffer...(MATCH)
*****************************************
Adding buffer (9371708 bytes) to MPQ using (0x8), level (0)...
Time to add: 2486.212074 msec
Reading file buffer from MPQ...
Time to read: 342.787142 msec
Testing read buffer...(MATCH)

I know, it's 1.8, the special WinMPQ ver, but I think it doesn't matter. It shows the difference quite clearly.

I doesn't look that impressive. But you have to take into account that that is the 3000+ on Socket 754 (2GHz, 512KB L2). Current Socket 939 models have dual-channel memory. Additionally I believe that it is possible to optimize that code even further for K8.

BTW, can you give me the specs of the machine you did the benches in the readme?
Starcraftfreak is offline  
Old 03-26-2005, 03:39 AM   #13
BlacKDicK
User


Respected User
 
Join Date: Mar 2002
Posts: 589

BlacKDicK is on a distinguished road (24)

Default

Quote:
Originally Posted by Starcraftfreak
BTW, can you give me the specs of the machine you did the benches in the readme?
Sure, here we go:
Athlon XP 2000+ 1666Mhz (Thorton, 266 FSB, 256KB L2)
A7V266-E
1x Samsung 512MB DDR 400 (@ 266Mhz 2,2,2,5)
GF4MX400
2x PATA Samsung 120GB (7200 rpm, 2MB) @ MOBO RAID-0

Nothing unusual on it, the memory at low latency and the RAID-0 help general performance bit.

Thanks for the info. I will clean up the code a bit and if you´re interested, I can send it to you; maybe you can try some SSE2 or K8 stuff on it.

Last edited by BlacKDicK : 03-26-2005 at 03:49 AM.
BlacKDicK is offline  
Old 03-26-2005, 11:54 AM   #14
Vexorian
Free Software Terrorist
 
Vexorian's Avatar


Technical Director
 
Join Date: Apr 2003
Posts: 14,898

Submissions (37)

Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)Vexorian has a reputation beyond repute (1062)

Hero Contest #3 - 2nd Place

Default

I got delphi 2005, when it is so easy to get new software as in this country, you don't need to know what you are exactly buying.

But anyways it allows to make .net apps, but the problem is that the current SFMPQ.dll is not too friendly with .net argument wise, you should try using some .net program and sfmpq header to see what I mean.

Since you are the only guy that seems to be updating it, could you try to get a .net compatible version?, no rush needed, just 2 or 3 years before a new game comes out, There is no need for an app to be .net at this time, but it will sooner or later get like that
__________________
Zoom (requires log in)Wc3 map optimizer 5.0
Someone should fix .wav sound in this thing.
Zoom (requires log in)JassHelper 0.A.2.A
Turns your simple code into something that is complicated enough to work.
Faster != more useful
Vexorian is offline  
Old 03-26-2005, 03:21 PM   #15
Soar
User
 
Join Date: Feb 2003
Posts: 52

Submissions (1)

Soar is on a distinguished road (18)

Default

.net assembler is much slower than 32bit original assembler codes, that's why I only install win32 part of d2005 on my pc
Soar is offline  
Closed Thread


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off


All times are GMT. The time now is 04:08 PM.


Affiliates
The Hubb The JASS Vault Clan WEnW Campaign Creations Clan CBS GamesModding Flixreel Videos

Powered by vBulletin (Copyright ©2000 - 2020, Jelsoft Enterprises Ltd).
Hosted by www.OICcam.com
IT Support and Services provided by Executive IT Services