Home World Forum
Stars! AutoHost web forums

Jump to Stars! AutoHost


 
 
Home » Stars! Clones, Extensions, Modding » Stars! Extensions » Open-source M and H file merger tool available
Open-source M and H file merger tool available Tue, 01 September 2015 23:09 Go to next message
schneck is currently offline schneck

 
Crewman 3rd Class

Messages: 8
Registered: April 2015
Location: United States
I have taken Raptor's fine work on open-sourcing Stars! file decryption and produced a tool. The tool will allow allies to share information directly in their H or M files.

Java code at https://github.com/raptor/stars

Download runnable jar file at https://github.com/tupelo-schneck/stars/releases/download/v0 .9/StarsFileMerger.jar

It's been tested, but please backup your files if you use it, just in case! And let me know (via GitHub is best) what problems you encounter.

If you know not-yet-public information about the Stars! file structures that could help produce more tools like this, please do send me such information!

Using the tool:

java -jar StarsFileMerger.jar -m file...
java -jar StarsFileMerger.jar -h file...

StarsFileMerger will merge data among allies' M files or H files.
Use -m to merge M files, -h to merge H files.

If merging M files:
All M files supplied on the command line will have their data augmented
with the data on each planet, player, design, fleet, minefield, packet,
salvage, or wormhole from any of the files.

Backups of each input M file will be retained with suffix .backup-m#.

If merging H files:
All H files supplied on the command line will have their data replaced
with the newest data on each planet, player, and design from any of the files.

M files supplied on the command line will have their data incorporated
but will not be changed. M files are needed for accurately determining
the latest ship designs.

Backups of each input H file will be retained with suffix .backup-h#.

Usage notes:
If allies merge M files each turn, merging H files should not be necessary.
If any ally skips a turn, merging H files might become necessary.
H file merge can also be used for a single player to fill a new H file
from a collection of M file backups.

Report message to a moderator

Re: Open-source M and H file merger tool available Wed, 16 September 2015 09:41 Go to previous messageGo to next message
vmanuel is currently offline vmanuel

 
Chief Warrant Officer 3

Messages: 187
Registered: October 2004
Location: Dallas, TX USA
It is the most amazingly awesome tool not only for allies in a game, but for hosting. It allows me as host to merge the .m files of every player, so I can see the entire board from one screen.

Useless, you say? Try figuring out how good a spread a player has without deep diving and exporting spreadsheets. Or looking at a glance on turn 1 to see where all the planets are located. SO useful!

Thanks again for your work!



Editor in Chief of the Kaynan Space News.
All Space, All The Time - Kaynan!
www.myhood.biz

Report message to a moderator

Re: Open-source M and H file merger tool available Mon, 28 September 2015 11:49 Go to previous messageGo to next message
m.a@stars is currently offline m.a@stars

 
Commander

Messages: 2765
Registered: October 2004
Location: Third star to the left
schneck wrote on Wed, 02 September 2015 05:09
All M files supplied on the command line will have their data augmented with the data on each planet, player, design, fleet, minefield, packet, salvage, or wormhole from any of the files.

Neat! Cool

Would it be possible to just extract all that info to plain text files, perhaps in CSV (tab or comma-separated) format? I know of no tool that can easily extract ship designs, or minefield/packet/salvage/wormhole reports. Lurking

It would be handy to have a bunch of saved ship designs that could later be imported to other games instead of needing to be redesigned from scratch. Whip

It would also be super-handy to have planet-Queue and fleet data all neatly exported, too. Even if it couldn't later be imported. But if it could also be tweaked and then imported, the sky's the limit! Twisted Evil



So many Stars, so few Missiles!

In space no one can hear you scheme! Deal

Report message to a moderator

Re: Open-source M and H file merger tool available Mon, 28 September 2015 12:01 Go to previous messageGo to next message
m.a@stars is currently offline m.a@stars

 
Commander

Messages: 2765
Registered: October 2004
Location: Third star to the left
vmanuel wrote on Wed, 16 September 2015 15:41
It is the most amazingly awesome tool not only for allies in a game, but for hosting.


Awesome, and potentially game-changing. Rolling Eyes

Now JoaTs will be able to 'sell' their exclusive penscans, SDs and ISs their minefield knowledge, ITs their gate maps, and CAs their terra and hab info, not just as gossip or coordinate sets, but as actionable target-able hard data directly in your map. Twisted Evil

It would be easier if players could decide exactly what types and ranges of their own private data they wanted to 'sell', of course, but I'm sure it would be a lively 'market'. my 2 cents my 2 cents my 2 cents




So many Stars, so few Missiles!

In space no one can hear you scheme! Deal

Report message to a moderator

Re: Open-source M and H file merger tool available Tue, 29 September 2015 13:34 Go to previous messageGo to next message
schneck is currently offline schneck

 
Crewman 3rd Class

Messages: 8
Registered: April 2015
Location: United States
m.a@stars wrote on Mon, 28 September 2015 11:49

Would it be possible to just extract all that info to plain text files, perhaps in CSV (tab or comma-separated) format? I know of no tool that can easily extract ship designs, or minefield/packet/salvage/wormhole reports. Lurking

It would be handy to have a bunch of saved ship designs that could later be imported to other games instead of needing to be redesigned from scratch. Whip

It would also be super-handy to have planet-Queue and fleet data all neatly exported, too. Even if it couldn't later be imported. But if it could also be tweaked and then imported, the sky's the limit! Twisted Evil


The code on GitHub supplies a bare skeleton for just this sort of thing. So, with some work, someone could extract ship designs (etc) from an M-file into some human-readable format, and also provide some way of importing such a format into an X-file.

I did the minimum work required for the tool I was building, so, for instance, ship designs are mostly opaque. It can tell when one ally has seen a ship design that another ally hasn't, but it hasn't actually decoded the various parts that go into the ship. Also, I haven't thought at all about the structure of X-files.

If anyone is keen to do such work, I would be happy to advise.

Also, I'll repeat the request for any information people have about file structures. Old forum posts hint at information passed around behind the scenes. Between what Raptor figured out and trial and error, it can presumably be recreated, as evidenced by the tool I was able to make. But it would be nice to save some of the effort.

Report message to a moderator

Re: Open-source M and H file merger tool available Tue, 29 September 2015 14:57 Go to previous messageGo to next message
m.a@stars is currently offline m.a@stars

 
Commander

Messages: 2765
Registered: October 2004
Location: Third star to the left
schneck wrote on Tue, 29 September 2015 19:34
Old forum posts hint at information passed around behind the scenes. Between what Raptor figured out and trial and error, it can presumably be recreated

I gave Raptor all I had. The rest is supposed to be easy to guess, with encryption out of the way. Lurking

Who knows what non-public tools may have been created, and for which purposes. In fact, I hope some "sanity-check" tool can be built soon to detect and squelch the dangerous uses of file knowledge. Rolling Eyes



So many Stars, so few Missiles!

In space no one can hear you scheme! Deal

Report message to a moderator

Re: Open-source M and H file merger tool available Sun, 13 December 2015 11:23 Go to previous messageGo to next message
raptor is currently offline raptor

 
Chief Warrant Officer 1
Creator of StarsWine
Created StarsWine

Messages: 138
Registered: June 2014
The best references for file structures I found are in multiple files that I decompiled from PaulCr's utilities. The decompiled code is found here:

https://github.com/raptor/stars/tree/master/decompiled

The best resources are probably in the 'StarsHostCreator' and 'StarsHostCreator_StarsHostEditor' folders. But there's stuff scattered all around. Sadly, the other utils I've got were all written and compiled in languages that make it difficult to extract the data (I have no source code of any of them.)

Report message to a moderator

Re: Open-source M and H file merger tool available Fri, 19 February 2016 19:52 Go to previous messageGo to next message
Altruist is currently offline Altruist

 
Commander

Messages: 1068
Registered: August 2005
Location: Berlin
Thanks a lot for this tool.

My last game was quite a big game. This tool was a very easy way to update especially my h-file and that helped tremendously.

What we relaized a bit too late, though, how much even more it could had helped us to do our orders in a close alliance... well, perhaps next game.


And absolutely definetly it will help with hosting!


[Updated on: Fri, 19 February 2016 19:53]

Report message to a moderator

Re: Open-source M and H file merger tool available Sun, 25 February 2018 20:07 Go to previous messageGo to next message
talkingbologna is currently offline talkingbologna

 
Senior Chief Petty Officer

Messages: 86
Registered: November 2016
Location: 1947
I apologize in advance if this is the simplest thing in the world to do, but let's assume I've never seen a command prompt. It seems like there's a few steps missing in the suggested method. You're telling me to go to the store, and as an adult, I know that obviously that takes getting up, going to and starting my car, then driving. But if I'd never seen a car and didn't know I had legs...

I've read the Stars! FAQ telling how to create a game from the command line, and have seen the above command for using the StarsFileMerger tool, but the two don't seem connected nor can I figure out with what to replace "-m file..." Do I use the command prompt inside DOSBox's Win 3.11? "java -jar StarsFileMerger.jar -newbeg.m4" does nothing in that prompt or the one in Win7. "Java is not a legal command" or something. If I have to add or replace code in the standalone program Stars! DOSBox batch file, how to I access it and where do I put the line?

Thank you for making such a tool, though.

Report message to a moderator

Re: Open-source M and H file merger tool available Tue, 27 February 2018 09:57 Go to previous messageGo to next message
Ron is currently offline Ron

 
Commander
Forum Administrator
Stars! AutoHost Administrator

Messages: 1232
Registered: October 2002
Location: Collegedale, TN
talkingbologna wrote on Sun, 25 February 2018 20:07
I've read the Stars! FAQ telling how to create a game from the command line, and have seen the above command for using the StarsFileMerger tool, but the two don't seem connected nor can I figure out with what to replace "-m file..." Do I use the command prompt inside DOSBox's Win 3.11? "java -jar StarsFileMerger.jar -newbeg.m4" does nothing in that prompt or the one in Win7. "Java is not a legal command" or something. If I have to add or replace code in the standalone program Stars! DOSBox batch file, how to I access it and where do I put the line?

Thank you for making such a tool, though.


You'll need to get Java from http://www.java.com, download it and install it.

For the following, see the attached screenshot.

As you can see, in my DOSBox, I created a game in c:\games

In the screenshot further down, you'll see the real path to that folder, as far as my Windows 7 system is concerned. Your DOSBox location is probably different from mine, so you'll have to take that into account.

In the CMD black window part of the screenshot, at the top, you'll see that I 'cd' (change directory) into the real path of the Stars! games folder, then (optional) I used the 'dir' command to view the directory listing, then I start to run the Java jar file on one of the Stars! game's .m files. NOTE: In this setup, you will have to have the .jar file in the games folder for it to work.

Since I don't actually have that .jar program on my computer, I can't show what its output looks like.

Hope that helps.

(just noticed that I don't even have a .m4 file in that folder, oh well) Rolling Eyes
  • Attachment: sgshot.jpg
    (Size: 296.39KB, Downloaded 250 times)


[Updated on: Tue, 27 February 2018 09:59]




Ron Miller
Stars! AutoHost

Report message to a moderator

Re: Open-source M and H file merger tool available Fri, 09 March 2018 10:43 Go to previous messageGo to next message
ricks03 is currently offline ricks03

 
Officer Cadet 1st Year
Creator of TotalHost and Stars! utilities
Created TotalHost and Stars! utilities

Messages: 221
Registered: January 2012
Location: NC
The github links are dead. If anyone has the download, can you send me a copy or provide a link?


https://www.irelandbybicycle.com
http://totalhost.sinister.net:999
https://github.com/ricks03/TotalHost

Report message to a moderator

Re: Open-source M and H file merger tool available Fri, 09 March 2018 17:19 Go to previous messageGo to next message
raptor is currently offline raptor

 
Chief Warrant Officer 1
Creator of StarsWine
Created StarsWine

Messages: 138
Registered: June 2014
Hi,

We have since moved all the projects to a github group:

https://github.com/stars-4x/

All the utilities are found in this project, but you have to load it in the eclipse IDE and run the various classes:

https://github.com/stars-4x/starsapi

The latest code is in there, with bugfixes and additional features over the old. We'll graciously accept features and bugfixes, and if you want to be part of the project to develop it more, just ask.


Some of the old files can still be found here:

https://github.com/tupelo-schneck/stars/releases


[Updated on: Fri, 09 March 2018 17:22]

Report message to a moderator

Re: Open-source M and H file merger tool available Fri, 22 June 2018 23:52 Go to previous messageGo to next message
talkingbologna is currently offline talkingbologna

 
Senior Chief Petty Officer

Messages: 86
Registered: November 2016
Location: 1947
Ron Wrote

You'll need to get Java from http://www.java.com, download it and install it.

For the following, see the attached screenshot.

As you can see, in my DOSBox, I created a game in c:\games

In the screenshot further down, you'll see the real path to that folder, as far as my Windows 7 system is concerned. Your DOSBox location is probably different from mine, so you'll have to take that into account.

In the CMD black window part of the screenshot, at the top, you'll see that I 'cd' (change directory) into the real path of the Stars! games folder, then (optional) I used the 'dir' command to view the directory listing, then I start to run the Java jar file on one of the Stars! game's .m files. NOTE: In this setup, you will have to have the .jar file in the games folder for it to work.

Since I don't actually have that .jar program on my computer, I can't show what its output looks like.

Hope that helps.

(just noticed that I don't even have a .m4 file in that folder, oh well) Rolling Eyes


Eureka!!
I finally figured out how to merge .h and .m files. Seems the program needs the actual file names, not just the file type suffixes. Did you guys know that? Razz

I feel like the the cop on South Park who did six months of research to a rock montage to figure out that a left hand is the same one direction or the other.

Report message to a moderator

Re: Open-source M and H file merger tool available Sat, 23 June 2018 08:03 Go to previous messageGo to next message
mrvan is currently offline mrvan

 
Officer Cadet 1st Year

Messages: 220
Registered: May 2014
Just wanted to pop in and say that I absolutely love the merger tool. I've not gotten the hang of stars notebook and like to have my info in the main interface, so this really helps share intel with your ally.

I wrote a little python script that automatically gets the m files from the server and merges them. I set it to run as a cron job and publish them on a (pwd protected) website, but I could easily change it to mail the users their merged file or you could write them to a shared dropbox folder. https://github.com/vanatteveldt/starstools/blob/master/merge .py

Report message to a moderator

Re: Open-source M and H file merger tool available Wed, 15 August 2018 00:44 Go to previous messageGo to next message
ricks03 is currently offline ricks03

 
Officer Cadet 1st Year
Creator of TotalHost and Stars! utilities
Created TotalHost and Stars! utilities

Messages: 221
Registered: January 2012
Location: NC
can you point me to a link for the specific merge code that the .jar file is built from?


https://www.irelandbybicycle.com
http://totalhost.sinister.net:999
https://github.com/ricks03/TotalHost

Report message to a moderator

Re: Open-source M and H file merger tool available Sun, 17 February 2019 08:15 Go to previous messageGo to next message
mrvan is currently offline mrvan

 
Officer Cadet 1st Year

Messages: 220
Registered: May 2014
Hey guys,

First, thanks for making the merger utility. I've created an issue there, but I don't really know what is the best channel for communication so I thought I would also post here:


https://github.com/stars-4x/starsapi/issues/10

The file merger tool is a fantastic tool for sharing complete information between allies. However, I sometimes wish to share some information with another player, e.g. if we are waging a war together but I don't have a permanent alliance. So, I would like to be able to share e.g. all planet hab values and ownership, pop and defense% only for enemy planets, all ship locations, and ship designs only of enemy ships.

I had a look at the merger file source code and it looks fairly understandable. So, I would like to make two new tools:
Exporter - exports selected information from the .h file into a json (or similar) file
Importer - imports the exported json file into an .h file

Does that make sense? Does something like that exist? Does it indeed sound feasible to do something like that?

Report message to a moderator

Re: Open-source M and H file merger tool available Thu, 21 February 2019 22:08 Go to previous messageGo to next message
m.a@stars is currently offline m.a@stars

 
Commander

Messages: 2765
Registered: October 2004
Location: Third star to the left
mrvan wrote on Sun, 17 February 2019 14:15
Does that make sense? Does something like that exist? Does it indeed sound feasible to do something like that?

Oh, yes! Cool Not that I know of. Confused Please please yes! Twisted Evil

Might want to add enemy minefields while you're at it.

Also, consider filtering by distance or location, not just by race.



So many Stars, so few Missiles!

In space no one can hear you scheme! Deal

Report message to a moderator

Re: Open-source M and H file merger tool available Wed, 17 June 2020 15:34 Go to previous message
platon79 is currently offline platon79

 
Chief Warrant Officer 3

Messages: 184
Registered: February 2004
Location: Norway
raptor wrote on Fri, 09 March 2018 17:19
Hi,
We have since moved all the projects to a github group:
https://github.com/stars-4x/
All the utilities are found in this project, but you have to load it in the eclipse IDE and run the various classes:
https://github.com/stars-4x/starsapi
The latest code is in there, with bugfixes and additional features over the old.


About the "with bugfixes and additional features", I just published the current github code as a jar at https://github.com/stars-4x/starsapi/releases/download/Stars FileMerger_june2020/StarsFileMerger_june2020.jar so that thouse who need an up to date version (I just encountered a stars crash with the old version myself in my current game) can get it easy. Let us know if there are any issues.

Report message to a moderator

Previous Topic: Graphing player resources
Next Topic: Stars! AI
Goto Forum:
  


Current Time: Wed Feb 01 08:20:09 EST 2023