Home World Forum
Stars! AutoHost web forums

Jump to Stars! AutoHost


 
 
Home » Stars! Clones, Extensions, Modding » Stars! Extensions » My wheels are turning (A different approach to Stars! management)
icon3.gif  My wheels are turning Fri, 18 April 2014 03:53 Go to next message
CrazyBS is currently offline CrazyBS

 
Petty Officer 1st Class

Messages: 68
Registered: November 2010
Location: Portland, OR
I've been thinking. We have all wanted a new version of Stars! for awhile now and we all know by now that it is a lot of work to say the least. It's not something that gets a lot of steam because we all love the current version of Stars! and are busying playing it. The source code is also not public.

I have a new proposal:
To provide a way to submit a turn without the need of using the current "client". Preferably using a browser application with mobile/tablet friendly design.

Really it's just a matter of understanding the way the current game files are assembled, allowing creation or modifications to a turn using a visual interface, and reassembling those game files back again. Then you can upload them to Autohost.

By designing a tool with just this in mind, I can make an incremental step towards more integration of the original game. A battle viewer perhaps. Enhanced fleet tracking using the history of turns. Better MM management. Integration with Autohost to submit a turn. And the shining "star", allow the web server to run submitted turns through Stars! on the backend and replace the current Autohost (hopefully by enhancing Autohost to version 2).

The application will likely take the form of a Java/Spring application using JSON endpoints and a HTML/javascript frontend.

Let me know what you guys think. I am currently planning out the epics for this work in Jira now and I hope to have an Agile project moving forward with incremental deliverables each sprint that have an actual product owner and prioritization. I will make the stories public after this initial conversation and I come up with estimates of how much time that this will eat IRL.

Questions? Problems? Concerns?


[Updated on: Fri, 18 April 2014 04:11]




--------------------------
Brenton Pyl
Technical Lead
Catalyst IT Services, Inc.

Report message to a moderator

Re: My wheels are turning Fri, 18 April 2014 05:10 Go to previous messageGo to next message
m.a@stars is currently offline m.a@stars

 
Commander

Messages: 2765
Registered: October 2004
Location: Third star to the left
I understand you already have a way to read all data in the game's files and poke inside? That has been one of the hurdles for many people over the years. Sherlock


So many Stars, so few Missiles!

In space no one can hear you scheme! Deal

Report message to a moderator

Re: My wheels are turning Fri, 18 April 2014 10:37 Go to previous messageGo to next message
CrazyBS is currently offline CrazyBS

 
Petty Officer 1st Class

Messages: 68
Registered: November 2010
Location: Portland, OR
Well, that's the trick right?

I've spent some time last night going over the information we have regarding that and it seems there is some hesitation to allow the file formats to be shared. Fear of cheating I suppose...

I was thinking about this and it would be possible to create a non open source JAR that handles the file import/export and only exposes the information that the player should have, not the rest. The rest of the application would still be open source and available to anyone that wants to use it.

But that still requires that "someone" implements the java interface. I had hoped that we had enough information formed about the format of the files that this could be accomplished. Either by myself with help from those that have studied it or completely behind closed doors from me. The output would be the JAR that implements the interface.

I have seen a couple of "initiatives" to get this information, most notably this post: Stars! API. But that doesn't seem to have any links to the result or any source code, which makes work like this difficult. They even mention porting some code to Java.

I am hoping that if we avoid exposing the internals of the import/export, but output an implementation of a java class that can expose the player only details, that we can get those involved and the rest of the Stars! community's "blessing" on the tool. I certainly don't want to make cheating any easier, that's no the point of an application like this. The point is to allow us the ability to not be tied to the client to view and plan moves. We have all been in that position at lunch, really wishing we had a way to view a turn from our phone or tablet. Laptops help of course, but it is time we open up Stars! to a more accessible interface.




--------------------------
Brenton Pyl
Technical Lead
Catalyst IT Services, Inc.

Report message to a moderator

Re: My wheels are turning Fri, 18 April 2014 11:03 Go to previous messageGo to next message
skoormit is currently offline skoormit

 
Lieutenant

Messages: 665
Registered: July 2008
Location: Alabama
CrazyBS wrote on Fri, 18 April 2014 02:53


I have a new proposal:
To provide a way to submit a turn without the need of using the current "client". Preferably using a browser application with mobile/tablet friendly design.


The rest of your post indicates that you mean to provide a way to "create" or "play" turns, not "submit" turns. (I don't mean to quibble. But I do think that a project of this scope benefits from having as clear a mission statement as possible.)

It sounds like a lovely idea. And it sounds like a very complex undertaking (but you know that already).



What we need's a few good taters.

Report message to a moderator

Re: My wheels are turning Fri, 18 April 2014 12:53 Go to previous messageGo to next message
m.a@stars is currently offline m.a@stars

 
Commander

Messages: 2765
Registered: October 2004
Location: Third star to the left
CrazyBS wrote on Fri, 18 April 2014 16:37
Well, that's the trick right?

I've spent some time last night going over the information we have regarding that and it seems there is some hesitation to allow the file formats to be shared. Fear of cheating I suppose...

Indeed. And there's lots of info around, all in little bits and pieces, but if anyone has the complete picture they're not telling. Sherlock

This should have been a community project from the start, years ago. Instead all we have is several amazing solo efforts that have sputtered and all but vanished. Sad

That JAR of yours sounds like a neat solution, if it can be done. From there I'm guessing we could reconstruct most or even all of the files' internals, which would lead to all kinds of great tools, including anti-cheating. Cool



So many Stars, so few Missiles!

In space no one can hear you scheme! Deal

Report message to a moderator

Re: My wheels are turning Fri, 18 April 2014 21:14 Go to previous messageGo to next message
CrazyBS is currently offline CrazyBS

 
Petty Officer 1st Class

Messages: 68
Registered: November 2010
Location: Portland, OR
Quote:
The rest of your post indicates that you mean to provide a way to "create" or "play" turns, not "submit" turns. (I don't mean to quibble. But I do think that a project of this scope benefits from having as clear a mission statement as possible.)


Yeah, I agree with that. I'll clarify that, we certainly want to be able to work on a turn without using the client.

Quote:
From there I'm guessing we could reconstruct most or even all of the files' internals, which would lead to all kinds of great tools, including anti-cheating. Cool


Since the submission of turn data would be in JSON form to the server, there would be a requirement to ensure that the turn data passes sanity tests. We would know about the race of the player and their current research levels/technology through the race file and the M file. So it's all theoretically possible.

The main point of contention is that we will need to be able to get to and from a valid Stars! file to a representation of the data. The idea is this:

public interface StarsGeneratedTurn
{
  // Could be streams
  public StarsGeneratedTurn(byte[] fileData); 

  // Represents the same byte[] array passed in.  This is the .M file
  public byte[] tokenize(); 

  // StarsYearData would be a POJO filled with all the information needed to view the yearly details
  public StarsYearData getYearData();
}

public interface StarsTurn
{
  // Creates a new turn definition from a generated turn
  public StarsTurn(StarsGeneratedTurn generatedTurn); // Creates a new turn definition from a generated turn

  // Opens up a previously tokenized turn.
  // I'm not sure if we actually need the generated turn here, but could be useful for validation rules
  public StarsTurn(StarsGeneratedTurn generatedTurn, byte[] tokenizedTurn); 

  // This will be the .X file representing the current state of the turn.
  public byte[] tokenize(); 

  // StarsTurnData is a simple POJO with a collection of turn information
  public StarsTurnData getTurnData();

  // The provided data would have edits, additions or deletions from the getTurnData().
  // Validation 
  public List<ValidationErrors> setTurnData(StarsTurnData turnData); 

  // Validates a turn and returns the results
  public List<ValidationErrors> validateTurn(StarsTurnData turnData);
}


As a first stab at it.. that looks pretty good. Now we just have to get the right people to agree to implement it or provide the information to me to do so myself.

The more I think about this project, the more this would be better as a "upgrade" to SAH version 2.0. Mainly adding additional features that allow the players to click on their turn and view it, edit it and submit when they are satisfied. That way the sensitive source code for these files doesn't get into the wrong hands. However, with the JSON representation being the interface to the data, I can build on top of those interfaces and build a useful app to compliment SAH.



--------------------------
Brenton Pyl
Technical Lead
Catalyst IT Services, Inc.

Report message to a moderator

Re: My wheels are turning Sat, 19 April 2014 04:36 Go to previous messageGo to next message
m.a@stars is currently offline m.a@stars

 
Commander

Messages: 2765
Registered: October 2004
Location: Third star to the left
CrazyBS wrote on Sat, 19 April 2014 03:14
there would be a requirement to ensure that the turn data passes sanity tests. We would know about the race of the player and their current research levels/technology through the race file and the M file. So it's all theoretically possible.

I'm guessing historical data on past turns and even "universe status" will be needed too. Sherlock


Quote:
The main point of contention is that we will need to be able to get to and from a valid Stars! file to a representation of the data.

Indeed. I've had many ideas over the years about neat things that could be built, including "playing w/out the client" but so far I've only been able to turn a handful of these ideas into practical tools. Wall Bash



So many Stars, so few Missiles!

In space no one can hear you scheme! Deal

Report message to a moderator

Re: My wheels are turning Sun, 20 April 2014 19:04 Go to previous messageGo to next message
CrazyBS is currently offline CrazyBS

 
Petty Officer 1st Class

Messages: 68
Registered: November 2010
Location: Portland, OR
Well, the first iterations of the web tools will be a storage of data. Game settings, race settings, Planet, Fleet and the .map. This will at least give a way to view and browse through basic information that we already have using the offline tools like Stars! Notebook and such, but will allow for persistent history. I've never seen a good fleet tracker for Stars! yet that really takes advantage of history, which we will have available.

Ideally, I'd like to be able to generate this information from a running game. Without the export/import of raw Stars! files, I can "fake" it by allowing you to upload your .XY and your .M# and I will shove that into a Stars! client running in an emulator or something. That will only give me access to the basic text files we already have access to, but without the need to generate the reports yourself. I could even imagine the SAH providing those two files to the tool auto-magically using existing protocols...

The tool in this iteration will basically be online versions of the Stars! Notebook and potentially the Stars! Calculator with a simple to access interface (hopefully).

The second iteration I hope to be able to view and update orders and queues, build ships and starbases, read and write messages and adjust research settings with the goal of producing a .X that can be used in the Stars! client. All of these are only possible with a conversion from a .m#/.x to a JSON of what we need to know.

Given those idea, I am working on organizing the user stories and level of effort it would take to get us to the first milestone of an integrated online tool for just the report data. Once the framework is in place, I want to open it up to the few on the forums that want to participate in coding.

The goal at any step is to always have something useful that can be used and demoed. The Stars! rewrites in the past have all tried the "waterfall" technique of build everything from the ground up before it can be used. I hope to ensure that this project has deliverables that can be used at every iteration of the software, so when it eventually runs out of steam, there is something useful left behind.

But, at this point I'm just full of hot air and ideas and still need to define the goals so that I can get to work on a base framework.

I'm thinking a Java/Spring backend with a Mongo database. I'm not sure if I want to go Angular on the frontend yet, it seems like overkill.

m.a@stars - What is your level of interest in this idea? Would you want to get involved at some point?



--------------------------
Brenton Pyl
Technical Lead
Catalyst IT Services, Inc.

Report message to a moderator

Re: My wheels are turning Mon, 21 April 2014 02:40 Go to previous messageGo to next message
XAPBob is currently offline XAPBob

 
Lt. Commander

Messages: 957
Registered: August 2012
Can do better than the basic text files already http://starsautohost.org/sahforum2/index.php?t=msg&th=5 324&start=0&rid=1560
(fixed url)


[Updated on: Tue, 22 April 2014 09:33] by Moderator


Report message to a moderator

Re: My wheels are turning Mon, 21 April 2014 07:54 Go to previous messageGo to next message
m.a@stars is currently offline m.a@stars

 
Commander

Messages: 2765
Registered: October 2004
Location: Third star to the left
CrazyBS wrote on Mon, 21 April 2014 01:04
m.a@stars - What is your level of interest in this idea? Would you want to get involved at some point?

I'm up for brainstorming at any time, but my knowledge of JSON/Ajax is almost nil, despite most of my tools being written in JavaScript. Sherlock



So many Stars, so few Missiles!

In space no one can hear you scheme! Deal

Report message to a moderator

Re: My wheels are turning Mon, 21 April 2014 11:49 Go to previous messageGo to next message
m.a@stars is currently offline m.a@stars

 
Commander

Messages: 2765
Registered: October 2004
Location: Third star to the left
http://starsautohost.org/sahforum2/index.php?t=msg&th=53 24&start=0&rid=1560


So many Stars, so few Missiles!

In space no one can hear you scheme! Deal

Report message to a moderator

Re: My wheels are turning Mon, 21 April 2014 12:58 Go to previous messageGo to next message
XAPBob is currently offline XAPBob

 
Lt. Commander

Messages: 957
Registered: August 2012
This very forum, message pane scraper

Report message to a moderator

Re: My wheels are turning Mon, 21 April 2014 15:27 Go to previous messageGo to next message
skoormit is currently offline skoormit

 
Lieutenant

Messages: 665
Registered: July 2008
Location: Alabama
CrazyBS wrote on Mon, 21 April 2014 10:17
XAPBob wrote on Sun, 20 April 2014 23:40
Can do better than the basic text files already -http://starsautohost.org/sahforum2/index.php?t=msg&th=5 324&start=0&rid=1560


That link doesn't seem to work even when I manually throw it together. Where is that XAPBob?


Here



What we need's a few good taters.

Report message to a moderator

Re: My wheels are turning Tue, 22 April 2014 20:55 Go to previous messageGo to next message
CrazyBS is currently offline CrazyBS

 
Petty Officer 1st Class

Messages: 68
Registered: November 2010
Location: Portland, OR
oooooohhhh...

It's an EXE tool.. but that won't stop me. I wonder if I XyliGUN would be willing to part with the source code for that baby so I can port into Java, otherwise I'll have to run it in a command prompt or something...

The existence of that tool makes me think that this is a valid endeavor and I can start playing with visualizing a turn. I'll work on actually "playing" a turn later.

As this topic indicates, my wheel are turning...



--------------------------
Brenton Pyl
Technical Lead
Catalyst IT Services, Inc.

Report message to a moderator

Re: My wheels are turning Tue, 22 April 2014 21:53 Go to previous messageGo to next message
LittleEddie is currently offline LittleEddie

 
Lieutenant
Helped track down one or more Stars bugs

Messages: 517
Registered: February 2011
Location: Delaware
Yes, I call the scraper tool and then load that data, I have a AR/IS mixed engine fleet fuel usage tool I'm working on but spring has come and it will be 6 months before I can put a lot of time into it again. It works, just not very well.

My coding is rusty right now as I took a few years off, but I plan on starting again in October. No business, No for Profit, I just want to code again for the fun of it.

Report message to a moderator

Re: My wheels are turning Wed, 23 April 2014 04:27 Go to previous messageGo to next message
m.a@stars is currently offline m.a@stars

 
Commander

Messages: 2765
Registered: October 2004
Location: Third star to the left
LittleEddie wrote on Wed, 23 April 2014 03:53
I have a AR/IS mixed engine fleet fuel usage tool I'm working on but spring has come and it will be 6 months before I can put a lot of time into it again. It works, just not very well.

Sounds interesting! Mind giving more detail? Sherlock



So many Stars, so few Missiles!

In space no one can hear you scheme! Deal

Report message to a moderator

Re: My wheels are turning Wed, 23 April 2014 07:28 Go to previous messageGo to next message
LittleEddie is currently offline LittleEddie

 
Lieutenant
Helped track down one or more Stars bugs

Messages: 517
Registered: February 2011
Location: Delaware
m.a@stars wrote on Wed, 23 April 2014 04:27
LittleEddie wrote on Wed, 23 April 2014 03:53
I have a AR/IS mixed engine fleet fuel usage tool I'm working on but spring has come and it will be 6 months before I can put a lot of time into it again. It works, just not very well.

Sounds interesting! Mind giving more detail? Sherlock



Just using the standard fuel usage math but with the scrapper you can.
(Stars! Trip and Fuel Usage Calculator by M.A.2004)

1. Select Fleet & add/subtract ships.(gives X, Y, Fuel, Cargo space)
2. Adjust Pop/mineral levels.(start with whatever is in the fleet)
3. Select end point.(Planet, Fleet, XY)
4. Calc the optimize jumps.
5. Break the fleet down by ship type.(only 16 possible, easier then grouping by engine).
6. For each ship type
6a. Do one year travel.
6b. Adjust Pop & Fuel(adding or subtracting as needed).
6c. Do again until you reach the end point.
7. Put the fleet back together
8. Did we have enough fuel?

That's basically it, a lot of details left out.

Ed

Report message to a moderator

Re: My wheels are turning Wed, 23 April 2014 07:43 Go to previous messageGo to next message
XAPBob is currently offline XAPBob

 
Lt. Commander

Messages: 957
Registered: August 2012
Random SS "hack" occurs...

Take Boranium in a freighter into enemy space (or lift it with a RB) and load into any part loaded freighters you can see...

They run out of fuel and take longer to deliver their cargo, and where did all that boranium in my hold come from?

Report message to a moderator

Re: My wheels are turning Wed, 23 April 2014 09:54 Go to previous messageGo to next message
skoormit is currently offline skoormit

 
Lieutenant

Messages: 665
Registered: July 2008
Location: Alabama
LittleEddie wrote on Tue, 22 April 2014 20:53
No business, No for Profit, I just want to code again for the fun of it.


This is how I know I am with my people.



What we need's a few good taters.

Report message to a moderator

Re: My wheels are turning Wed, 23 April 2014 09:57 Go to previous messageGo to next message
skoormit is currently offline skoormit

 
Lieutenant

Messages: 665
Registered: July 2008
Location: Alabama
XAPBob wrote on Wed, 23 April 2014 06:43
Random SS "hack" occurs...

Take Boranium in a freighter into enemy space (or lift it with a RB) and load into any part loaded freighters you can see...

They run out of fuel and take longer to deliver their cargo, and where did all that boranium in my hold come from?



ROFLMAO ROFLMAO ROFL ROFL

There are now two ways to make war by giving away boranium.



What we need's a few good taters.

Report message to a moderator

Re: My wheels are turning Wed, 23 April 2014 13:13 Go to previous messageGo to next message
m.a@stars is currently offline m.a@stars

 
Commander

Messages: 2765
Registered: October 2004
Location: Third star to the left
LittleEddie wrote on Wed, 23 April 2014 13:28
1. Select Fleet & add/subtract ships.(gives X, Y, Fuel, Cargo space)
2. Adjust Pop/mineral levels.(start with whatever is in the fleet)
3. Select end point.(Planet, Fleet, XY)
4. Calc the optimize jumps.
5. Break the fleet down by ship type.(only 16 possible, easier then grouping by engine).
6. For each ship type
6a. Do one year travel.
6b. Adjust Pop & Fuel(adding or subtracting as needed).
6c. Do again until you reach the end point.
7. Put the fleet back together
8. Did we have enough fuel?

That's basically it, a lot of details left out.

Ahhh. Much more complete than my little tool. I should probably add that kind of functionality to mine, but for some reason the need never arose. Whip

I'm guessing you "optimize" jumps to save fuel, maximize distance, and such? Sherlock



So many Stars, so few Missiles!

In space no one can hear you scheme! Deal

Report message to a moderator

Re: My wheels are turning Thu, 24 April 2014 11:47 Go to previous messageGo to next message
LittleEddie is currently offline LittleEddie

 
Lieutenant
Helped track down one or more Stars bugs

Messages: 517
Registered: February 2011
Location: Delaware
For any given distance there's a best speed, so on a trip that takes more then one turn you can break it down into 'legs' so each turn is done at a different speed. Say a trip that takes 3 years (175lys), your options are, warp

9 + 9 + 9 (1873mg test ship)(default)
9 + 9 + 4 (1735mg)
9 + 8 + 6 (1428)
8 + 8 + 7 (1146)
9 + 7 + 7 (1361)

Which one uses the least fuel depends on the engine type (by slot) and total weight (by slot) of the fleet.

Report message to a moderator

Re: My wheels are turning Fri, 30 May 2014 03:24 Go to previous messageGo to next message
mhermans is currently offline mhermans

 
Civilian

Messages: 1
Registered: August 2013
[quote title=m.a@stars wrote on Fri, 18 April 2014 12:53]CrazyBS wrote on Fri, 18 April 2014 16:37

This should have been a community project from the start, years ago. Instead all we have is several amazing solo efforts that have sputtered and all but vanished. Sad


I can't shake the feeling this is closely linked with with the closed source approach in this community.

For instance, I had started on a Python API for turn generation and information extraction. Nothing fancy, just a personal itch to scratch and an idea for a weekend hobby-programming project.

However, looking through the forum and old mailinglists was extremely demotivating. If there has been no effort towards reverse-engineering and extending, it would be not so bad, a challenge to discover things first. But now you need to thrawl through years of dead initiatives and vague ideas, and realise that you are wasting those scarce weekend programming hours on repeating effort.

For instance, this API-thread is baffeling. A closed source, invite-only project to develop an API. Dead after two years, all effort apparently lost.

And it is not that the information on password decryption, file structure, etc. is not available somewhere. Patched EXE's have been floating around for decades, people "in the know" have shared and independently reversed-engineerd the encryption and (largely) the file format. So you know that both other programmers and cheaters have that advantage, it is just you that are stuck when you want to write something.

But as a lurker I can't go around demanding what people or the community does with their code/effort. I'm just going to keep visiting this forum every half a year, looking at how the community dwindles, previous knowledge dissapears, until hopelfully some day the fear of a dead community overtakes the fear of cheaters.

On a positive note: if you are going for a fully open source project, I'm willing to pitch in time/code working on the JSON-interface.

Report message to a moderator

Re: My wheels are turning Fri, 30 May 2014 06:35 Go to previous messageGo to next message
m.a@stars is currently offline m.a@stars

 
Commander

Messages: 2765
Registered: October 2004
Location: Third star to the left
mhermans wrote on Fri, 30 May 2014 09:24
For instance, this API-thread is baffeling. A closed source, invite-only project to develop an API. Dead after two years, all effort apparently lost.

Not that there was a lot of effort, IIRC, it was just an attempt to summarize in one place all publicly available info. But you're right, almost all efforts have been doomed for fear of promoting cheating, despite everyone knowing that it's the cheating you don't know that kills you. Evil or Very Mad

I've written many tools to help with MM, intel-gathering, etc, only to be stopped by the decryption barrier. Why bother building something for others to use if it won't be publishable? Nevermind what amazing tools other ppl could have come up with if they had had easy access to turn files. Whip

So, if the problems are the cheating, or the faulty server, let's attack those 1st, either with anti-cheating filters, as discussed in "message pane" scraper (last page) and elsewhere, or by rebuilding the entire server from scratch, as discussed in Stars 3, which is more work, but is perhaps more glamorous and has lots of other potential. Help welcome! Cheers



So many Stars, so few Missiles!

In space no one can hear you scheme! Deal

Report message to a moderator

Re: My wheels are turning Fri, 30 May 2014 09:00 Go to previous messageGo to previous message
XAPBob is currently offline XAPBob

 
Lt. Commander

Messages: 957
Registered: August 2012
The problem is that without access to the work already done any progress made towards either of those admirable goals will surely stall through frustration at doing stuff that you *know* someone else could just tell you.

Report message to a moderator

Previous Topic: m and h files merger tool
Next Topic: "message pane" scraper
Goto Forum:
  


Current Time: Thu Mar 28 14:32:22 EDT 2024