Home » Stars! Clones, Extensions, Modding » Stars! Extensions » "message pane" scraper
| "message pane" scraper |
Fri, 26 July 2013 13:12  |
|
 XyliGUN |
 | |
Messages: 325
Registered: July 2004
Location: Russia, St.Petersburg | 
|
|
Several peoples asked me by mail is it possible to dump messages from m file.
Instead of answer to each of you let me just start a new topic here on forum.
First of all there are 3 "types" of messages in Stars:
1. server messages, which are part of the m file (like fleet X has been dismantled for XkT of minerals at the starbase orbiting Planet Y.)
2. client messages (like, you just found planet X) - these are not stored in m file, but generated by client based on data from m and h files.
3. in-game messages between players.
1 - can be extracted easily.
2 - needs to be generated based on comparision of data from m & h files
3 - can be extracted, but there are some minor issues related to non-english letters (like russian or german).
Second question is the form, in which you would like to have these messages?
Messages in Stars!.exe are stored like this "\\F has been dismantled for \\vkT of minerals at the starbase orbiting \\p."
m file just contain a message id and message args, which can be planet id, design id, fleet id or number of pop/minerals etc.
All of this can exported to csv, xml, txt, but which one would you like, depends of how are you going to use it.
Any hints on this?
Also there are several options regarding security and password:
1. it can be used by player itself in that case tool can be public, but it should verify password
2. it can be used by SAH only and in that case it will ignore password protection, since SAH cannot know players passwords
Comment, suggestions are welcome! 
"Progress isn't made by early risers. It's made by lazy men trying to find easier ways to do something."
Robert A. Heinlein, Time Enough For LoveReport message to a moderator
|
|
|
| | | | | | | |
| Re: "message pane" scraper |
Thu, 01 August 2013 13:29  |
|
| XyliGUN |
| |
Messages: 325
Registered: July 2004
Location: Russia, St.Petersburg |
|
|
XAPBob wrote on Wed, 31 July 2013 21:19Thinking on this a little more....
One of my issues with stars! is that even to do a simple -dpf it requires a graphical display.
The Jeffs were economical with bandwidth, but maybe not so much with compute resources (which are now emulated).
Would it be possible to have something that generated a CSV (with quoted strings where appropriate) version of the p and f files, along with a similar 'msg' file. These three could trivially be imported into a spreadsheet for manipulation.
Currently I have to kick off a remote X session in order to extract this data, which rather limits where I can do so...
I guess I can export values that are stored inside m file for you, but there is also some data (like planet value) which is not stored, but calculated.
And this one is not so simple, but having planet stat you can calculate planet value in something like Excel.
"Progress isn't made by early risers. It's made by lazy men trying to find easier ways to do something."
Robert A. Heinlein, Time Enough For LoveReport message to a moderator
|
|
|
| | | |
| Re: "message pane" scraper |
Thu, 01 August 2013 18:34 |
|
| XyliGUN |
| |
Messages: 325
Registered: July 2004
Location: Russia, St.Petersburg |
|
|
m.a@starsThere's the small problem of what to do with all the info in the m files that players aren't supposed to know, like CA-learnt hab values and such. 
But that could be solved by inserting a "filter" tool between what Stars! gens and what SAH lets players download. 
Regarding the issue with the CA having full opponents race block (which includes for example, all of the race file settings, all of the current tech levels, current research points, current research priorities, available MT toys and other stuff) in his M file...
Good news is that the problem is not that big as I thought before.
- While in M files generated by Stars! 2.6J RC3 and 2.7J RC3 that data is present, and can be pretty easily extracted if you know decryption and the blocks structure.
- But Stars! 2.6J RC4, which is used by SAH, does not have this problem at all! In case of M file generated by 2.6J RC4 that block is still present in CA m file, but all of the data except hub values are zeroed.
I cannot find a turn generated by 2.6J RC4 with the MT inside to check whether it is also zeroed or not.
Please, if you have one with the MT, send it to me for research.
"Progress isn't made by early risers. It's made by lazy men trying to find easier ways to do something."
Robert A. Heinlein, Time Enough For LoveReport message to a moderator
|
|
|
| | | | | | |
| Re: "message pane" scraper |
Wed, 07 August 2013 06:25 |
|
| XyliGUN |
| |
Messages: 325
Registered: July 2004
Location: Russia, St.Petersburg |
|
|
m.a@stars* There's a couple "extra" bytes at start. Are they for formatting or versioning purposes?
Seems like you mean UTF8 header bytes?
m.a@stars* Including the full "message string" at the end of each line seems a bit overkill.
Sure. It's temporary here just for the debugging purposes and to make it easy to understand what does all these numbers means. But for an automation purposes EventId column should be used.
m.a@stars* I can't remember what the "HintId" column is for. 
Currently I always make it 0, but in fact there is a two byte "shortcut" used by goto button and it can reference planet, fleet, technology browser, etc. I'm not sure if it have any usage for automation purposes, but in any case I have to sort it out before dumping into the file.
m.a@stars* The "recent breakthrough" messages seem to have a different format for some params. 
Well, yes and no. Format is the same, but in fact several event params actually use two value args, this is why I currently put them in one Arg, separated by "|" symbol. I'm still thinking on how to correctly dump these ones.
m.a@starsPerhaps you'll like to test against much bigger (and complex) m-files? I should have a handful fat ones still around. 
Absolutely, just need to make sure it works on all of my games collection.
"Progress isn't made by early risers. It's made by lazy men trying to find easier ways to do something."
Robert A. Heinlein, Time Enough For LoveReport message to a moderator
|
|
|
| |
| Re: "message pane" scraper |
Wed, 07 August 2013 10:05 |
|
| XyliGUN |
| |
Messages: 325
Registered: July 2004
Location: Russia, St.Petersburg |
|
|
Here is short description on what will be stored for each of the event param in CSV file. Hope this will make things a bit more clear.
\E - environment area value, stored in form Mesure|Value (measures: 0 - g, 1 - °C, 2 - mR, value... seems to be in clicks from the left???)*
\F - ship id, stored in format: DesignId|FleetId
\g or \G - currently exported as an integer value, but it wasn't found in any of the m files
\I - change type (0 = decreased, 1 = increased)
\j - zero based research area id (0 - energy, 1 - weapons, 2 - propulsion, 3 - construction, 4 - electronics, 5 - biotechnology)
\k - technology id, stored in format: GroupId|ItemId (group examples: armour, engines, orbital, terraforming, etc.)*
\l, \L or \Z - zero based player id
\M - mine field type (0 - standard, 1 = heavy, 2 - speed bump)
\n - location, can be in form of either planet id, fleet id or space coordinates
planet is stored in form P|PlanetId
fleet is stored in form F|OwnerId|FleetId
space coordinates is stored in form C|x|y
\O - zero based owner id (the same as player id)
\P - percent value
\S - packet origin (the same as player id)
\V - just an integer value, but comparing to \v it always used as is, since it is already multiplied by 100
\v - just an integer value (note: some events treat it as is, some add '00'/'kT'/'mg')
\X - strange, but \X is always replaced by empty string in any event message, will be stored as empty Arg in export
\z - design id, stored in format: IsStarbase|DesignId (IsStarsbase values: 0 = false, 1 = true)
* Notes:
- Two lines highlighted in orange - this is where community input would be more than welcome.
- \E - this value is always stored as integer number and in case of temperature it is "as is" equal to temperature value shown in Stars! But I don't know how to convert this integer value to floating point gravity value and to radiation value. I cannot find it now, but if I remember correctly I have seen somewhere on the forum, how does this value can be converted.
- \k - I don't have an group/item ids mapping to the technologies/parts at the moment, so it would be nice if someone can collect it later on.
"Progress isn't made by early risers. It's made by lazy men trying to find easier ways to do something."
Robert A. Heinlein, Time Enough For LoveReport message to a moderator
|
|
|
| | |
| Re: "message pane" scraper |
Wed, 07 August 2013 11:23 |
|
| XyliGUN |
| |
Messages: 325
Registered: July 2004
Location: Russia, St.Petersburg |
|
|
m.a@starsXyliGUN\k - I don't have an group/item ids mapping to the technologies/parts at the moment, so it would be nice if someone can collect it later on.
Perhaps these numbers relate to the corresponding columns in the "allitems table" found inside the exe (or here) that I used as the basis for my (slightly more readable) table dumper?
Hmmm... partially yes, in this MOD file first number could be group id, and the second one is an item id.
Item id seems be exactly the same, except in MOD file it's 1-based, and in event it is 0-based.
As of group id, it seems to be always a power of 2 and it not corresponds to first number as well as power itself is not also corresponds to the first number.
Here is a few examples:
11,6,"Bear Neutrino Barrier" in MOD corresponds to 4|5 in m file. 5 is zero based, so we have to increment it by 1 and 5 + 1 = 6. But group id is not matched.
5,15,"Temp Terraform ±11" in MOD corresponds to 8192|14. 14 is again a zero based, so 14 + 1 = 15 - matched. But group id is not matched.
6,12,"Laser Battery" in MOD corresponds to 32768|11. 11 + 1 = 12 - matched, but group id is not.
"Progress isn't made by early risers. It's made by lazy men trying to find easier ways to do something."
Robert A. Heinlein, Time Enough For LoveReport message to a moderator
|
|
|
|
Goto Forum:
Current Time: Sun Mar 26 05:40:22 EDT 2023
|
Members
Search
Help
Register
Login
Home
