Object Data Extractor (Alpha)

[PurplePoot]

Originally Posted by Rising_Dusk

What is the advantage of not initializing them right on map init anyways? What sort of delay/lag is experienced when loading the data for a lot of types?

One hashtable save per unit per field. If I recall correctly, you were the one who initially requested the feature (although it may have been EF). More importantly, the problem with priority on map initialization is also a reason.

Originally Posted by Rising_Dusk

Have you made it work properly for the case where multiple libraries request the same data field?

Then it would just load that field once.

Originally Posted by Rising_Dusk

Furthermore, let's say a map only needs access to, say, damage base or armor base. Will they be able to access just those fields without loading all of the data for the types?

The tool only loads the fields you ask for (as parameters to LoadUnitData). Perhaps I misunderstand your question?

[Deaod]

Looks awesome.
Gonna toy a bit around with it.

Btw, whatll happen to levelable fields (in abilities)?

Edit: Hrm, so this is run before JassHelper, right?
Thats gonna cause some problems with the ObjectMerger and GMSI. Maybe generate valid JASS code directly and run this after JH?

[ploks]

Originally Posted by Rising_Dusk

Eh, I probably wouldn't use it for a gameplay constant extractor. Really, the constants you can just declare as constant globals and be done with it. It's not like unit data where 100 unit types each have a different value for X field.

I guess you are right. I just hate to manually change the agility -> armour if I would like to do a balance change. But maybe your extra work is too much for my little convenience. ;)

[grim001]

Originally Posted by PurplePoot

More importantly, the problem with priority on map initialization is also a reason.

If you initialize the data within a struct onInit, and you place that struct within the "ExportedUnitData" library, then it will definitely be initialized before any library that requires ExportedUnitData.

[Rising_Dusk]

Originally Posted by PurplePoot

The tool only loads the fields you ask for (as parameters to LoadUnitData). Perhaps I misunderstand your question?

No, it seems you didn't. I'm just trying to verify that the tool doesn't do any unnecessary work and only what is asked of it. Glad to see that it does.

Originally Posted by PurplePoot

One hashtable save per unit per field. If I recall correctly, you were the one who initially requested the feature (although it may have been EF). More importantly, the problem with priority on map initialization is also a reason.

So all things considered, it's actually quite fast. And yeah, I did request it, but that doesn't mean it's a good solution necessarily. I'm just making sure it's useful. I think it is. I imagine for most custom scripts, the -noinit won't be used, but that's fine.

[PurplePoot]

Originally Posted by Deaod

Edit: Hrm, so this is run before JassHelper, right?
Thats gonna cause some problems with the ObjectMerger and GMSI. Maybe generate valid JASS code directly and run this after JH?

Doesn't JH run pJASS? If so, that isn't an option. Also, what problems does this have with ObjectMerger and GMSI (other than it missing the object data that ObjectMerger adds the first time you save with that tool)?

Originally Posted by Rising_Dusk

So all things considered, it's actually quite fast. And yeah, I did request it, but that doesn't mean it's a good solution necessarily. I'm just making sure it's useful. I think it is. I imagine for most custom scripts, the -noinit won't be used, but that's fine.

It only took about 30 seconds (literally) to write, so if a single person finds it useful then it's worth it.

Originally Posted by Rising_Dusk

No, it seems you didn't. I'm just trying to verify that the tool doesn't do any unnecessary work and only what is asked of it. Glad to see that it does.

Indeed. It doesn't even process the MetaData of any unrequested fields, nor load any unnecessary SLKs. However, it does load every Profile file if a single one is needed, since as far as I can tell there is no good way to tell which ones will be without a search which would just take as much if not more time. There really isn't any excuse to load unnecessary data, though, since although everything is bizarrely crossreferenced it is at least descriptive.

Originally Posted by grim001

If you initialize the data within a struct onInit, and you place that struct within the "ExportedUnitData" library, then it will definitely be initialized before any library that requires ExportedUnitData.

Neat. However, once again there is a priority issue (structs with onInit) which becomes a problem, and seeing as onInit methods in structs are probably a more useful place to have code involving this than library initializers I doubt that would fix more. I'll certainly look for other solutions, though (for example, if pJASS is or can be split from JH then writing native code would be an option).

[Rising_Dusk]

Isn't there any way to avoid having to make a library require UnitExportData? It's very lame that it has to do that.

Also, Mr.Malte raised a good point.

JASS:

    local UnitDataStruct up = GetUnitDataStruct(u)
    call BJDebugMsg(R2S(up.minDamage))
    call BJDebugMsg(R2S(up.maxDamage))
    call BJDebugMsg(I2S(up.armor))
    call BJDebugMsg(R2S(up.attackRate))
    call BJDebugMsg(up.model)

This is faster than:

JASS:

    call BJDebugMsg(R2S(GetUnitMinDamage(u)))
    call BJDebugMsg(R2S(GetUnitMaxDamage(u)))
    call BJDebugMsg(I2S(GetUnitArmor(u)))
    call BJDebugMsg(R2S(GetUnitAttackRate(u)))
    call BJDebugMsg(GetUnitModel(u))

Is there any way to support this struct method too?

[grim001]

Originally Posted by PurplePoot

Neat. However, once again there is a priority issue (structs with onInit)

Struct onInits for structs within libraries run in the same order that the libraries compile. As I said, if you put your init stuff within a struct onInit within the ExportedObjectData library, it will definitely be done before any code is executed from any library that requires ExportedObjectData. Also, structs that are not inside any library run their onInit methods after structs inside libraries are done. That means there would be no issues with struct onInits within scopes or outside of any library/scope.

It would be even better if you just inserted the initialization call into the Main function before anything else gets done, and made sure the GetUnitType* functions are declared above all libraries. That would let us drop the library requirement. But I'm not sure if that would be easy for you to do.

[Anitarf]

Originally Posted by PurplePoot

No. Do you have a reason to? I could implement that relatively easily, but I just didn't see a need. It could also add some ambiguity (what if one had -noinit and the other didn't, for example?).

This was likely already clarified, but I'll state it again for the sake of clarity:

Libraries using this system should, if at all possible, be able to declare what data they need within themselves, rather than it being specified within oyur library. The clear advantage of this is that users can just copy a library that uses this system into their map and have it working, rather than being required to edit your system's calibration section for each library they import that uses it (or, the lamer alternative, to simply load all data instead of bothering to eliminate that which is not needed from the list).

If things are done this way, steps should be taken to ensure you don't get double loading if two libraries require the same value.

Not sure what you mean by the -noinit thing, the data should get initialized before it is used automatically, library requirements should ensure that.

Originally Posted by Rising_Dusk

Isn't there any way to avoid having to make a library require UnitExportData? It's very lame that it has to do that.

I don't see the issue. Why would UnitExportData (or whatever this will be called in the end) be any different from any other requirement?

Also, Mr.Malte raised a good point.

JASS:

    local UnitDataStruct up = GetUnitDataStruct(u)
    call BJDebugMsg(R2S(up.minDamage))
    call BJDebugMsg(R2S(up.maxDamage))
    call BJDebugMsg(I2S(up.armor))
    call BJDebugMsg(R2S(up.attackRate))
    call BJDebugMsg(up.model)

This is faster than:

JASS:

    call BJDebugMsg(R2S(GetUnitMinDamage(u)))
    call BJDebugMsg(R2S(GetUnitMaxDamage(u)))
    call BJDebugMsg(I2S(GetUnitArmor(u)))
    call BJDebugMsg(R2S(GetUnitAttackRate(u)))
    call BJDebugMsg(GetUnitModel(u))

Is there any way to support this struct method too?

Most libraries only require to read a single value, in which case encapsulating the whole thing in a struct would hurt performance. In the few cases where you'd need multiple values (such as for a triggered attack engine that would need to read several of the attack data fields such as range, arc, projectile art, damage...) you can always encapsulate the data in a struct specific to that system yourself.

[PurplePoot]

Originally Posted by Rising_Dusk

Isn't there any way to avoid having to make a library require UnitExportData? It's very lame that it has to do that.

Also, Mr.Malte raised a good point.

JASS:

    local UnitDataStruct up = GetUnitDataStruct(u)
    call BJDebugMsg(R2S(up.minDamage))
    call BJDebugMsg(R2S(up.maxDamage))
    call BJDebugMsg(I2S(up.armor))
    call BJDebugMsg(R2S(up.attackRate))
    call BJDebugMsg(up.model)

This is faster than:

JASS:

    call BJDebugMsg(R2S(GetUnitMinDamage(u)))
    call BJDebugMsg(R2S(GetUnitMaxDamage(u)))
    call BJDebugMsg(I2S(GetUnitArmor(u)))
    call BJDebugMsg(R2S(GetUnitAttackRate(u)))
    call BJDebugMsg(GetUnitModel(u))

Is there any way to support this struct method too?

Because that speed difference is so critical (see my response to Ani's response to this post)?

I could support the struct method, but I do not see any need to and it's kinda nice to have some form of standardization. Besides, the current method resembles the natives moreso than structs (which allows them to stick out slightly less), and consistency is nice. Finally, people are way too obsessed with OOP; not everything has to be an object!

Originally Posted by grim001

Struct onInits for structs within libraries run in the same order that the libraries compile. As I said, if you put your init stuff within a struct onInit within the ExportedObjectData library, it will definitely be done before any code is executed from any library that requires ExportedObjectData. Also, structs that are not inside any library run their onInit methods after structs inside libraries are done. That means there would be no issues with struct onInits within scopes or outside of any library/scope.

Alright, good to know.

Originally Posted by grim001

It would be even better if you just inserted the initialization call into the Main function before anything else gets done, and made sure the GetUnitType* functions are declared above all libraries. That would let us drop the library requirement. But I'm not sure if that would be easy for you to do.

It's not so much what is easy for me to do as what blows up when I do it before JH edits the script. In this case, I'm pretty sure that inject would screw this over, although if you were using inject you would presumably know that already.

Originally Posted by Anitarf

This was likely already clarified, but I'll state it again for the sake of clarity:

Libraries using this system should, if at all possible, be able to declare what data they need within themselves, rather than it being specified within oyur library. The clear advantage of this is that users can just copy a library that uses this system into their map and have it working, rather than being required to edit your system's calibration section for each library they import that uses it (or, the lamer alternative, to simply load all data instead of bothering to eliminate that which is not needed from the list).

If things are done this way, steps should be taken to ensure you don't get double loading if two libraries require the same value.

Not sure what you mean by the -noinit thing, the data should get initialized before it is used automatically, library requirements should ensure that.

Already implemented in 1.1. An unlimited number of such calls are allowed, and double+ requests for the same value are simply ignored beyond the first.

Originally Posted by Anitarf

I don't see the issue. Why would UnitExportData (or whatever this will be called in the end) be any different from any other requirement?

Indeed. If anything, requiring that line makes it clear that your system needs this tool, which is half the use of requires (notice how several people campaigned for requires on scopes for this exact reason).

PS: It's going to stay as ExportedUnitData. The only reason I changed it from ExportedObjectData is so that you can (if you want) have units/items/etc separate, which may make sense and if nothing else forces some clarity into the library requirements.

Originally Posted by Anitarf

Most libraries only require to read a single value, in which case encapsulating the whole thing in a struct would hurt performance. In the few cases where you'd need multiple values (such as for a triggered attack engine that would need to read several of the attack data fields such as range, arc, projectile art, damage...) you can always encapsulate the data in a struct specific to that system yourself.

Indeed. If you are trying to load it every 0.04 seconds in 200 different missiles at the same time (the only time at which you could possibly notice such a minor speed difference), you're most likely doing it wrong. However, I doubt you would even notice it then unless you were making a whole bunch of lookups.

People are way too obsessed with efficiency for the sake of efficiency these days.

EDIT: I should also mention that structs would probably be slower than hashtables for this, for both reading and writing.

[Rising_Dusk]

Originally Posted by Anitarf

Most libraries only require to read a single value, in which case encapsulating the whole thing in a struct would hurt performance. In the few cases where you'd need multiple values (such as for a triggered attack engine that would need to read several of the attack data fields such as range, arc, projectile art, damage...) you can always encapsulate the data in a struct specific to that system yourself.

It'd still be faster if it were preloaded into structs for users who want it that way, since then the user would never use the hashtable calls. This would work best perhaps with a modifier argument to the external call like follows:

JASS:

//! LoadUnitData [-noinit] [-struct] fld1 fld2 fld3 fld4 fld5

If you don't want it that way, don't use the modifier. As Poot said, if it helps one person and there's a reason you'd want to do it that way (a system that needs many calls and can't afford too many hashtable calls), then it's worth doing.

Originally Posted by Anitarf

I don't see the issue. Why would UnitExportData (or whatever this will be called in the end) be any different from any other requirement?

It seems you misunderstood me, since you made the same argument I did in the paragraph before the above quote. My request is to avoid having to do the following:

JASS:

library A requires UnitExportData
//! LoadUnitData ...
endlibrary

And instead do;

JASS:

library A
//! LoadUnitData ...
endlibrary

I want to get rid of the library requirement altogether and just have the functions be before everything if that's at all possible.

Consider for an instance how lame it would be if any library that used ObjectMerger had to do the following:

JASS:

library A requires ObjectMergerData
//! external ObjectMerger ...
endlibrary

Not that it'd make sense to have it done that way, but it'd still be incredibly lame if that requirement were there.

[PurplePoot]

Originally Posted by Rising_Dusk

It'd still be faster if it were preloaded into structs for users who want it that way, since then the user would never use the hashtable calls. This would work best perhaps with a modifier argument to the external call like follows:

JASS:

//! LoadUnitData [-noinit] [-struct] fld1 fld2 fld3 fld4 fld5

If you don't want it that way, don't use the modifier. As Poot said, if it helps one person and there's a reason you'd want to do it that way (a system that needs many calls and can't afford too many hashtable calls), then it's worth doing.

There are a few differences between this and noinit:

• This would take a lot more than 30s to implement.
• The ambiguity if some libraries request this and some don't is a serious issue, whereas for noinit it is probably trivial.
• Loading all the data into structs would be much slower than loading it into hashtables.
• Reading the data from structs would be much slower than reading it from hashtables, and would in fact probably be two hashtable lookups in the end.

Originally Posted by Rising_Dusk

Consider for an instance how lame it would be if any library that used ObjectMerger had to do the following:

JASS:

library A requires ObjectMergerData
//! external ObjectMerger ...
endlibrary

Not that it'd make sense to have it done that way, but it'd still be incredibly lame if that requirement were there.

That's obviously different; in this case you are actually requesting a library of functions.

By that argument the requires keyword is lame in general. I could always make it automatically add that to a library's requirements, but I'm not sure whether I want to (again, see clarity).

[Rising_Dusk]

Originally Posted by PurplePoot

That's obviously different;

It's really not that different. What if the ObjectMerger had special functions associated with it? It'd still be lame to require an extra library.

Originally Posted by PurplePoot

in this case you are actually requesting a library of functions.

Can we not request the library of functions? Can we access the hashtables directly?

Originally Posted by PurplePoot

This would take a lot more than 30s to implement.

So have countless other features requested of me for my systems. I complain, but I still do it (if it makes sense).

Originally Posted by PurplePoot

Loading all the data into structs would be much slower than loading it into hashtables.

How does it take more work to populate a struct than it does to populate a hashtable? That makes no sense whatsoever.

Originally Posted by PurplePoot

The ambiguity if some libraries request this and some don't is a serious issue, whereas for noinit it is probably trivial.

Not if all libraries that need anything get their own copy of the data, which makes sense to sate Anitarf's earlier request anyways.

Originally Posted by PurplePoot

Reading the data from structs would be much slower than reading it from hashtables, and would in fact probably be two hashtable lookups in the end.

Uh, no? How does reading from a struct (which just does an array lookup) cost the same as 2 hashtable calls?

[PurplePoot]

Originally Posted by Rising_Dusk

It's really not that different. What if the ObjectMerger had special functions associated with it? It'd still be lame to require an extra library.

Well, if a good amount of people agree with you then I'll do the automatic adding of it.

Originally Posted by Rising_Dusk

Can we not request the library of functions? Can we access the hashtables directly?

I could make it that way, but they keys are pretty arbitrary. I don't really see any need to add a whole bunch of extra variables just so that people can bypass the (useful) wrappers. Also, as I said in my reply to the first quote, if a good amount of people agree with you then I will make the tool automatically add requirements if they are not present.

Originally Posted by Rising_Dusk

So have countless other features requested of me for my systems. I complain, but I still do it (if it makes sense).

I was just listing ways in which it was different from -noinit. I'm certainly implementing other features which take a lot more than 30s.

Originally Posted by Rising_Dusk

How does it take more work to populate a struct than it does to populate a hashtable? That makes no sense whatsoever.

Because beyond the initializing of every struct instance etc, you will still need hashtable calls (see below).

Originally Posted by Rising_Dusk

Not if all libraries that need anything get their own copy of the data, which makes sense to sate Anitarf's earlier request anyways.

Ani's request is already sated by simply making multiple calls to LoadUnitData allowed.

Originally Posted by Rising_Dusk

Uh, no? How does reading from a struct (which just does an array lookup) cost the same as 2 hashtable calls?

Sorry, it would only cost a single hashtable call (brain fart on my part), plus an additional function call. Why? Because the only good ways to implement a struct interface, at least as far as I can think of, would be one of:

JASS:

globals
    hashtable hash
endglobals

struct fields
    integer damagebase1
    //whatever else they requested. These are all set at map init.
endstruct

struct UnitData
    static method [] takes integer uid returns fields
        return LoadInteger(hash,uid,0)
    endmethod
endstruct

//or (using fields again)...

function UnitData takes integer uid returns fields
    return LoadInteger(hash,uid,0)
endfunction

Saving would look similar.

[Rising_Dusk]

Somehow this Mr.Malte character has done the struct method with GMSI and without hashtables here.

Originally Posted by PurplePoot

if a good amount of people agree with you then I will make the tool automatically add requirements if they are not present.

This works the same way as eliminating the library requirement from the libraries that do require it, making it as portable as ObjectMerger and satisfying my request. I say do it.