Wc3C.net

Wc3C.net (http://www.wc3c.net/forums.php)
-   Misc. Tutorials (http://www.wc3c.net/forumdisplay.php?f=430)
-   -   Adding your own Natives to warcraft III (c/c++) (http://www.wc3c.net/showthread.php?t=84417)

PipeDream 06-18-2006 08:28 PM

Adding your own Natives to warcraft III (c/c++)
 
2 Attachment(s)
In this tutorial I'm going to show how to use xttoc's jAPI to write your own warcraft natives. You'll need to know C/C++ and have a compiler whose calling convention you know (IE, can get lucky with). I'll be using borland's bcc 5.60. Unfortunately I couldn't get anything free to work.

Theoretically with this technique we'll be able to do anything at all, although it may take quite a bit of research. Even then there'll be a serious limitation in that maps won't be bnet distributable. I'd like to specifically warn against someone writing a general library that maps could call, as this would be begging for security problems.

In any case, let's get started. One thing that would be particularly nice to have is C routines for data structures. We'll implement two: One, a heap- just dynamically allocated arrays. Then we'll do something a little fancier, a disjoint set for mazes.
First off grab jAPI Tool from this thread:http://www.wc3campaigns.net/showthread.php?t=79652. Stuff all the files into your warcraft3 directory except for the jNatives folder-keep the folder but toss its contents. You'll probably want shortcuts for both the loaders, in particular, add the -window flag to the LoaderWar3.exe for testing, as in target: "D:\games\Warcraft III\LoaderWar3.exe" -window

The outline of what needs to happen is:
- Write a native
- Load native into warcraft
- Get custom declaration of native into common.j
- Write code that uses native
- Run game

We'll start with just a hello world. Here's our test.cpp:
Code:

/*
The following routines are all exported by japi.dll:

char* MemStrSearch(unsigned char *start, unsigned char *end, char *str);
bool MemPatternCompare(unsigned char *address, unsigned char *pattern, unsigned char *mask, unsigned long length);
unsigned char* MemPatternSearch(unsigned char *start, unsigned char *end, unsigned char *pattern, unsigned char *mask, unsigned long length);

PIMAGE_SECTION_HEADER GetImageSectionHeaders(HMODULE hModule, WORD *count);
PIMAGE_SECTION_HEADER GetImageSectionHeader(HMODULE hModule, unsigned char name[8]);

void        jAPI jBindNative(void *routine, char *name, char *prototype);
void        jAddNative(void *routine, char *name, char *prototype);
jString jAPI jStrMap(char *str);
char*        jAPI jStrGet(jString strid);
*/
// - Andy Scott aka xttocs


#include <windows.h>
#include <stdio.h>

#define jNATIVE        __stdcall
#define jAPI        __msfastcall

#define FloatAsInt(f) (*(long*)&f)
#define IntAsFloat(i) (*(float*)&i)

typedef long jString;
typedef long jInt;
typedef long jReal;

typedef void        (jAPI *jpAddNative)(void *routine, char *name, char *prototype);
typedef jString (jAPI *jpStrMap)        (char *str);
typedef char *        (jAPI *jpStrGet)        (jString strid);

jpAddNative                jAddNative;
jpStrMap                jStrMap;
jpStrGet                jStrGet;

#pragma warning ( disable : 4996 )


jString jNATIVE test(jString js, char *fnname)
{
        char *s;
        s = jStrGet(js);
        s[0] = 'A';
        return jStrMap(s);
}

BOOL APIENTRY DllMain(HMODULE hModule, DWORD  ul_reason_for_call, LPVOID lpReserved)
{
        if (ul_reason_for_call == DLL_PROCESS_ATTACH)
        {
                DisableThreadLibraryCalls(hModule);

                HMODULE hjApi = GetModuleHandle("japi.dll");

                jAddNative        = (jpAddNative)GetProcAddress(hjApi, "jAddNative");
                jStrMap                = (jpStrMap)*GetProcAddress(hjApi, "jStrMap");
                jStrGet                = (jpStrGet)*GetProcAddress(hjApi, "jStrGet");

                jAddNative(test, "test", "(S)S");
        }
    return TRUE;
}


The two important bits here are:
Code:

jString jNATIVE test(jString js, char *fnname)
{
        char *s;
        s = jStrGet(js);
        s[0] = 'A';
        return jStrMap(s);
}

and
Code:

                jAddNative(test, "test", "(S)S");
If stuff crashes and you're not using borland's compiler, try using __fastcall instead of __msfastcall.
Our test function takes a string and returns a string. It simply substitutes the first character in the string for the letter 'A', hoping that it wasn't passed a string of zero length. The engine also passes the name of the called function to the native so we include that fnname to make sure the stack gets cleaned up when we're done.
That third argument of jAddNative gives the type information. In parentheses comes the arguments. For example, (ISI) would mean takes integer x, string s, integer y. After the parenthesis comes the return type, in our case, S for string.
We need to compile a DLL. In a command prompt:
Code:

bcc32 -WD -e"test.xjp" test.cpp
-WD means build a dll, -e is the output file and test.cpp is of course the input. The file extension is .xjp instead of .dll because this is what jAPI recognizes and links into warcraft. Now stuff the .xjp file into where jAPI will look, the jNatives folder in your warcraft directory (which should be otherwise empty).
Next we need to add the native to common.j. If you don't have a copy, extract it with your favorite mpq archiver from war3patch.mpq.
Collapse JASS:
native test takes string s returns string
Now start LoaderWorldEditor.exe from the jAPI archive. If nothing at all happens, you may have registry issues. Try reinstalling warcraft. If it crashes, then something is probably wrong with the calling convention you set the functions to. Post the error message and the dll you generated and I can take a look. Now that the editor is running, create a new map and do whatever description fiddling you like. Import your modified common.j and change its base directory from war3mapImported\ to Scripts\ so that it overwrites the default blizzard copy. Save. Usually it'll fail the first time, before common.j imports properly. If after a couple saves you continue to get registered native declaration problems, check that the types you set in common.j match the types you set in the jAddNative call in test.cpp.

If everything is still going OK, write some JASS to try out your new test function. Try running call BJDebugMsg(test("Hello")) a few seconds into the game. Save your map. If the native doesn't seem to be declared, check that your common.j has the prototype line. If it saves ok, don't bother with the test map button. Instead run the jAPI LoaderWar3.exe, preferably through a shortcut with the -window flag. Head over to your map and start it up. If, upon clicking the map in the map selection screen, no player slots appear, you're probably missing some natives in your .xjp file. All the natives you added in common.j must have an entry in your custom dll.

If the game runs and displays "Aello", congratulations, you've written your first custom native. Now let's do something more interesting. One common lament of JASS programmers is the lack of dynamically allocated memory. We can't even pass arrays! In C, however, we can easily malloc() a new chunk of memory. This is a little dangerous, since it won't be cleaned up when the map finishes. We'll have to do it manually for now, but this is good practice anyway. For simplicity, and since we have the return bug for flexibility, we'll just write arrays of integers. So we're going to want four functions:
Collapse JASS:
native ArrayAlloc takes integer length returns integer
native ArrayFree takes integer arr returns nothing
native ArrayGet takes integer arr, integer index returns integer
native ArraySet takes integer arr, integer index, integer val returns nothing
Go ahead and add them to common.j

Next copy test.cpp to array.cpp and replace the test native with some array natives:
Code:

jInt jNATIVE ArrayAlloc(jInt len, char *fnname)
{
        return (jInt)malloc(sizeof(jInt)*len);
}

jInt jNATIVE ArraySet(jInt array,jInt index,jInt val, char *fnname)
{
        int *p = (int *)array;
        p[index] = val;
        return p[index];
}

jInt jNATIVE ArrayGet(jInt array, jInt index, char *fnname) {
        int *p = (int *) array;
        return p[index];
}

jInt jNATIVE ArrayFree(jInt array, char *fnname) {
        int *p = (int *)array;
        free(p);
        return 0;
}

I've had all of them return something since JASS seems to expect some sort of return value from every function, even if it's told that it returns nothing.
To register them, we'll need these lines in DllMain:
Code:

                jAddNative(ArrayAlloc,"ArrayAlloc","(I)I");
                jAddNative(ArraySet,"ArraySet","(III)I");
                jAddNative(ArrayGet,"ArrayGet","(II)I");
                jAddNative(ArrayFree,"ArrayFree","(I)I");


Compile this with bcc32 -WD -e"array.xjp" array.cpp
Toss out your test.xjp and replace it in the jNatives folder with array.xjp.

Try some simple test code, perhaps:
Collapse JASS:
local integer heap = ArrayAlloc(16)
call ArraySet(heap,5,42)
call BJDebugMsg(I2S(ArrayGet(heap,5)))
call ArrayFree(heap)
If you have problems, go back and check the troubleshooting tips listed for 'test'.
Exercise: Add bounds checking and dynamic resizing to your array.

Here's another native, this one's for blu and his maze generation map, a disjoint set:
Code:

typedef struct {
        int parent;
        int rank;
} node;

jInt jNATIVE DJSNew(jInt len, char *fnname)
{
        node *set = (node *)malloc(sizeof(node)*len);
        int i;
        for(i=0;i<len;i++) {
                set[i].parent = -1;
                set[i].rank = 0;
        }
        return (int)set;
}

jInt jNATIVE DJSFind(jInt iset, jInt x, char *fnname) {
        node *set = (node *)iset;
        if (set[x].parent == -1) return x;
        set[x].parent = DJSFind(iset,set[x].parent);
        return set[x].parent;
}

jInt jNATIVE DJSUnion(jInt iset,jInt x,jInt y, char *fnname)
{
        node *set = (node *)iset;
        int xr = DJSFind(iset,x);
        int yr = DJSFind(iset,y);
        if(set[xr].rank > set[yr].rank)
                set[yr].parent = xr;
        else if(set[xr].rank < set[yr].rank)
                set[xr].parent = yr;
        else {
                set[yr].parent = xr;
                set[xr].rank++;
        }
        return 0;
}

jInt jNATIVE DJSFree(jInt iset, char *fnname) {
        free((node *)iset);
        return 0;
}


Or some lists for weaaddar:
Code:

typedef struct {
        int car;
        int cdr;
} pair;

pair *allocpair() {
        return (pair *)malloc(sizeof(pair));
}

void freepair(pair *p) {
        free(p);
}

jInt jNATIVE cons(jInt x, jInt y, char *fnname) {
        pair *newpair = allocpair();
        newpair->car = x;
        newpair->cdr = y;
        return (jInt)newpair;
}

jInt jNATIVE car(jInt ipair, char *fnname) {
        pair *p = (pair *)ipair;
        return p->car;
}
jInt jNATIVE cdr(jInt ipair, char *fnname) {
        pair *p = (pair *)ipair;
        return p->cdr;
}


Prototype letters:
Code:

Integer                I
Real                R
String                S
Code                C
Boolean                B
Returns nothing        V
Handles                H; (?)
Handle ext.        H followed by ext followed by semicolon e.g. Hplayer;


Big thanks to xttocs for doing all the hard work and thanks for reading. If you manage to get this working with different compilers or how to work with handles or anything really, please post what/how! Good luck.

--------
Known good compiler list
Borland 5.60
Borland 5.5 (free)
--------
Important update: Each native needs an additional "char *fnname" argument at the end. Warcraft passes the name of the called native in so we need to make sure the stack is cleaned up. Thanks xttocs.
If you have problems with the stock loader, try the one in loader.rar

MindWorX 06-18-2006 10:29 PM

Tested, and approved (Not that my approval means much, but tested it, and i know it works)

Guesst 06-19-2006 01:39 AM

I got stuck here:
Quote:

Import your modified common.j and change its base directory from war3mapImported\ to Scripts\ so that it overwrites the default blizzard copy. Save. Usually it'll fail the first time, before common.j imports properly. If after a couple saves you continue to get registered native declaration problems, check that the types you set in common.j match the types you set in the jAddNative call in test.cpp.
It keeps giving me about a hundred script errors about excepting one thing or another when I try to save. I used your test.cpp. Is there something else I could be doing wrong?

Vexorian 06-19-2006 01:55 AM

I think that you should also have that common.j in a scripts subfolder of war3's folder.

And make sure you didn't make syntax errors when adding the natives to common.j.


I am talking out of my ass right now, but what if you try to add a: type superarray extends handle
and then make those natives use superarray as argument / return value ?

hmnn I should probably take my mingw32 and try to make this work there and then test the monster myself.

--

I renamed the title, also I moved it to misc tutorials. It is not really a JASS tutorial so it wouldn't fit there

PipeDream 06-19-2006 03:53 AM

Getting a little type safety on the arrays would be nice. However I'm pretty happy with integers since we don't have to return bug. I think if we want type safety it should come from a language on top of jass. Speaking of the return bug, you can write proper typecast natives that do the typecasting in C. This'll compile to just move input to output, so zero cost and way faster than the return bug. We can also do it with out understanding handles.
Apparently borland has a free as in beer compiler. Let me know if it requires any tricks.

- attached a test map with all the ingredients to help folks narrow down the source of problems.
- Added prototype information

Vexorian 06-19-2006 04:27 AM

figured out gcc wouldn't work.

I don't want a free as in beer compiler, I don't want to compile things with propietary compilers, I also had some unfun experiences with borland's vision of c++ but well maybe one of these days I'll try.


I wouldn't have any problem in using integers, but I am corious as if using the type define in common.j would actually work.

BertTheJasser 06-19-2006 02:40 PM

I am not familiar with c++, so it may take some time till I actually get all the constant keys, but anyways. Great tut! +Rep

blu_da_noob 06-19-2006 03:31 PM

Quote:

Originally Posted by PipeDream
Here's another native, this one's for blu and his maze generation map, a disjoint set:


:D

I'll have a look at this stuff when I get back from holiday, but I'm not sure this will ever be useful for general modding, due to the lack of portability (unfortunately :/).

Ragnarok X 06-19-2006 10:10 PM

I can't compile this even changing __fastcall instead of __msfastcall.

I am using Dev-C++ that is a free program, i am having problems with the command prompt, because i don't know what is that and i don't know if Dev-C++ have it.

Please help me.

PD: You think that with borland c++ 5 this work?

Vexorian 06-20-2006 01:47 AM

Can't understand why would anyone use dev-c++ instead of code::blocks . Anyways it is surelly configured to use gcc (mingw32) as compiler by default, you need to get the borland compiler and then configure dev-c++ to use it instead

Ragnarok X 06-20-2006 04:31 AM

Quote:

Originally Posted by Vexorian
Can't understand why would anyone use dev-c++ instead of code::blocks .


Why you say that? (reply this, please)

Anyway thanks for the advice, but how i can change the compiler if i have the borland one?.

PitzerMike 06-20-2006 07:10 AM

You don't have to compile from the IDE if you don't know how to configure it.
You can compile from command line.

Cool tutorial btw.

PipeDream 06-20-2006 08:59 AM

Quote:

I wouldn't have any problem in using integers, but I am corious as if using the type define in common.j would actually work.
Yep, it works. Neat.
ex:
common.j
Collapse JASS:
//Custom junk
type arr                extends handle
//Dynamic memory
native ArrayAlloc        takes integer len returns arr
native ArraySet            takes arr heap, integer index, integer value returns integer
native ArrayGet            takes arr heap, integer index returns integer
native ArrayFree        takes arr heap returns integer
war3map.j
Collapse JASS:
function Trig_trynative_Actions takes nothing returns nothing
    local arr heap
    call TriggerSleepAction(1.0)
    set heap = ArrayAlloc(16)
    call BJDebugMsg(I2S(HtoI(heap)))
    call ArraySet(heap,0,5)
    call BJDebugMsg(I2S(ArrayGet(heap,0)))
    call ArrayFree(heap)
endfunction
native.cpp
Code:

                jAddNative(ArrayAlloc,"ArrayAlloc","(I)Harr;");
                jAddNative(ArraySet,"ArraySet","(Harr;II)I");
                jAddNative(ArrayGet,"ArrayGet","(Harr;I)I");
                jAddNative(ArrayFree,"ArrayFree","(Harr;)I");


Quote:

Can't understand why would anyone use dev-c++ instead of code::blocks
the code blocks source debugger is rather lacking. regardless I recommend sticking to command line and using a real text editor.

Quote:

I don't want to compile things with propietary compilers
Open watcom should work. If __fastcall isn't microsoft style then you can use a little assembly stub like:
Code:

long addnativewrap(void *routine, char *name, char *prototype) {
 __asm {
  mov ecx,routine
  mov edx,name
  push prototype
  mov eax,jAddNative
  jmp eax //Some compilers don't like jumping to things that aren't labels
 }
}


Ragnarok X 06-21-2006 05:41 PM

Quote:

Originally Posted by PipeDream
We need to compile a DLL. In a command prompt:
Code:

bcc32 -WD -e"test.xjp" test.cpp


How i can use a command prompt?, how i can acceses to it?, somebody can explain me please?

PipeDream 06-21-2006 06:08 PM

I assume you use windows. Start->Run->command.

PipeDream 06-26-2006 10:56 PM

1 Attachment(s)
A couple people had problems with the stock jAPI loader. Here's a pure vanilla implementation I snarfed off the web with source.
Ripped from http://www.codeproject.com/threads/winspy.asp
Hopefully source for the meat and potatoes of jAPI.dll is coming soon.
Also fixed a serious bug with stack handling-the natives needed another argument on the end.

Daelin 07-18-2006 08:54 PM

I've got a question: How deep can you influence the aspects of the game? I mean... Do you know of a method which can manipulate the attributes of an object? You know... stuff like determining and modificating an unit's armor, maximum HP/MP, and so on and so forth...

I do not really get how this whole system works (I lack advanced programming knowledge) so that is why I'm asking. Don't tell me that I am asking a stupid question, because I already know that, just giving it a try. :P And if not, is anyone capable of transfering a logarithm function from the C++ libraries to JASS? It would be great, because in some cases, such a function could prove very useful. :)

~Daelin

PipeDream 07-18-2006 09:07 PM

The things you mention would be better off modified in JASS. There is voodoo for getting SLK fields which I have not worked out.

Computing logarithms can be done in JASS: http://www.wc3jass.com/viewtopic.php?t=159
Doing floating point work outside of JASS is tricky because you need to make it cross platform. However, numerically intensive work, perhaps custom path finding algorithms, is the sort of thing that this would be good for, because it would not require any more reverse engineering.

log is neat because you don't have to resort to calculus right away, there are a bunch of algebraic tricks.

Vexorian 07-18-2006 09:07 PM

With this it is easy to add a logarithm native. Really easy in fact as long as it is for a mod/campaign/single player map

Quote:

Doing floating point work outside of JASS is tricky because you need to make it cross platform
But is the loader cross platform ?

Daelin 07-18-2006 09:17 PM

Actually I am using the logarithm to attempt to detect the armor of an unit... Yes, this will probably be used in my mod if I succeed. I guess I'll have to learn this... Hmm... I'll see...

Oh... SLK fields... that is correct. Well, good luck! I am completely lost at such stuff!!

~Daelin

PipeDream 07-18-2006 09:34 PM

Not to Mac, but cross platform includes between AMD and Intel or even between a P3 and a P4. Unless someone can establish that the floating point units behave _identically_, you will have to use Blizzard's floating point emulation library for arithmetic. Alternatively, you can write float<->double routines in integer arithmetic and do all the FPU work in double precision.
Blizz FP emu Doc

Single player stuff is of course okay.

Daelin 07-23-2006 12:24 PM

Ok... I got the compiler 5.5 you are using as well. I got the test file, and put it in its directory. Then I typed exactly the stuff you mentioned above "bcc32 -WD -e"test.xjp" test.cpp" in the command prompt, but it tells me that I have 7 errors (three paranthesis errors and four Declaration Syntax Error). Am I doing something wrong? Am I supposed to put japi.dll in the compiler directory? Cuz I did and nothing happened.

And I just thought at a super useful function, which would avoid a lot of trouble and complications: a wait function that works for values under 0.27 seconds. If we had that, timers would no longer be that necessary, and neither would handle linking be. Codes would be simpler and faster.

~Daelin

MindWorX 09-22-2006 09:36 AM

Did you set all the envoirement variables for the compiler?

Tastingo 09-19-2008 08:35 PM

Would it be possible to save/load files from a computer. Also maybe getting information such as time? (if you wanted the time of the day in the game to match the world's)

PitzerMike 12-05-2008 12:35 PM

Quote:

Originally Posted by Tastingo
Would it be possible to save/load files from a computer. Also maybe getting information such as time? (if you wanted the time of the day in the game to match the world's)


Yes, that is possible.

elfian 04-12-2009 07:14 PM

Well I downloaded the jAPI tool. To compile the test.cpp file into a .dll I had to do some changes. the fastcall thing and the void* cast when calling jAddNative. I'm using Dev-C++. Well everything went well. But when I started the LoaderWorldEditor.exe it was totally bugged. All the buttons under the menu were missing. The trigger editor was totally destroyed. So I closed it and tried to launch it normally. It was the same. In the end I had to reinstall my whole game, because I couldn't figure the problem. I'm using 1.21b. Anyone could help me?

D.V.D 04-13-2009 09:07 PM

What if you imported the file that contains the natives and the Japi? Won't it be able to be distributed across b.net then?

TriggerHappy 04-14-2009 12:00 AM

Quote:

Originally Posted by D.V.D
What if you imported the file that contains the natives and the Japi? Won't it be able to be distributed across b.net then?


No, I don't think you know how jAPI works.

z3r0th 05-24-2011 03:45 PM

Version
 
Sorry for ressurect the topic, but i can't make it work

Its supposed to work with witch patch?

I'm tried to make it work with 1.21a, 1.21b, 1,24e, 1,26a with no success.

All with the same problem, the WE can't save the map due to errors in the script
I imported the common.j with the scripts path, used the custom WE loader, and when I save the map, get an error like this:



I'm using Windows 7 64 bits, but i tested (in a virtual machine) with windows xp 32 bits also.


Thanks for any help!


All times are GMT. The time now is 11:12 PM.

Powered by vBulletin (Copyright ©2000 - 2020, Jelsoft Enterprises Ltd).
Hosted by www.OICcam.com
IT Support and Services provided by Executive IT Services