Home World Forum
Stars! AutoHost web forums

Jump to Stars! AutoHost


 
 
Home » Stars! Clones, Extensions, Modding » Stars! Nova - Open Discussion and Help » Turn files documentation
Turn files documentation Thu, 07 July 2011 06:28 Go to next message
paul_ik is currently offline paul_ik

 
Master Chief Petty Officer

Messages: 98
Registered: November 2009
Location: Belarus GMT+2

Hi guys,
have you agreed upon turn files format used in Nova, will you document it soon?
I've browsed development docs in wiki, but "Turn Files" article contains only general info.

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 06:37 Go to previous messageGo to next message
Aeglos is currently offline Aeglos

 
Chief Warrant Officer 1
Stars! Nova developer
Stars! Nova developer

Messages: 142
Registered: May 2011
Location: Chile
Thinking of making some tools already? Shocked

We are currently on it right now, so specs are not final.

Current focus on the matter is to clear up the .intel file (that the server sends to the clients) so that only precise and decently organized data is sent. Progress is coming along nicely, but there's still things to be done.

Discussion just now started on .order files (client to the server) since right now it kinda passes unwanted information, and we want to have the server not trust the clients at all, to minimize cheating since the project is open source and the files are not currently encrypted, and will likely not be anyways.

EDIT: Currently files are plain XML, if it's relevant. Not made using auto xml serialization, but rather a homebrew xml serialization devised by Daniel, we basically construct the XML by hand for each object.


[Updated on: Thu, 07 July 2011 06:40]

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 06:56 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
As Aeglos says these are in a state of flux right now and I suspect will change a lot over time as well before we get a "final" release with everything in it.

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 07:16 Go to previous messageGo to next message
m.a@stars is currently offline m.a@stars

 
Commander

Messages: 2760
Registered: October 2004
Location: Third star to the left
Aeglos wrote on Thu, 07 July 2011 12:37

EDIT: Currently files are plain XML, if it's relevant. Not made using auto xml serialization, but rather a homebrew xml serialization devised by Daniel, we basically construct the XML by hand for each object.

Isn't XML a bit overkill for the task, when something like old excel-style csv could perhaps be plenty enough? Sherlock



So many Stars, so few Missiles!

In space no one can hear you scheme! Deal

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 07:18 Go to previous messageGo to next message
paul_ik is currently offline paul_ik

 
Master Chief Petty Officer

Messages: 98
Registered: November 2009
Location: Belarus GMT+2

All right then Smile
I'm not sure about the ideas I'll have when the formats are ready, but to try and write some bots is a tempting thing Razz

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 07:19 Go to previous messageGo to next message
Aeglos is currently offline Aeglos

 
Chief Warrant Officer 1
Stars! Nova developer
Stars! Nova developer

Messages: 142
Registered: May 2011
Location: Chile
Yes it would suffice, but XML is much more readable which helps with debugging and tracking data. And it's not like it's too hard to code when C# already abstracts and interfaces XML pretty well.

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 07:22 Go to previous messageGo to next message
Aeglos is currently offline Aeglos

 
Chief Warrant Officer 1
Stars! Nova developer
Stars! Nova developer

Messages: 142
Registered: May 2011
Location: Chile
paul_ik wrote on Thu, 07 July 2011 07:18

All right then Smile
I'm not sure about the ideas I'll have when the formats are ready, but to try and write some bots is a tempting thing Razz


Bots as in AI?

We have code in place that runs AI already, it pretty basic for now; build a couple of things and sends scouts. I don't know what's the plan for the AIs as I haven't touched that yet, but custom AIs can be made, albeit in C#.

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 07:25 Go to previous messageGo to next message
paul_ik is currently offline paul_ik

 
Master Chief Petty Officer

Messages: 98
Registered: November 2009
Location: Belarus GMT+2

m.a@stars wrote on Thu, 07 July 2011 14:16

Aeglos wrote on Thu, 07 July 2011 12:37

EDIT: Currently files are plain XML, if it's relevant. Not made using auto xml serialization, but rather a homebrew xml serialization devised by Daniel, we basically construct the XML by hand for each object.

Isn't XML a bit overkill for the task, when something like old excel-style csv could perhaps be plenty enough? Sherlock


CSV is convenient if you have flat data, containing one-to-one relations, not the thing you will expect with stars. XML is more flexible - you can add elements here and there, perfect feature for an actively developed project.
So among textual formats I think XML is pretty nice choice.

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 07:36 Go to previous messageGo to next message
m.a@stars is currently offline m.a@stars

 
Commander

Messages: 2760
Registered: October 2004
Location: Third star to the left
Aeglos wrote on Thu, 07 July 2011 13:19

Yes it would suffice, but XML is much more readable which helps with debugging and tracking data.

Much more readable than a plain flat table with headers? I might be a bit biased by too much Excel, but... Deal


Quote:

And it's not like it's too hard to code when C# already abstracts and interfaces XML pretty well.

Good. But plain CSV is about as simple as it gets. Almost like it was designed to be dumped right into (and out of) a program's arrays. Rolling Eyes


[Updated on: Thu, 07 July 2011 08:01]




So many Stars, so few Missiles!

In space no one can hear you scheme! Deal

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 07:37 Go to previous messageGo to next message
m.a@stars is currently offline m.a@stars

 
Commander

Messages: 2760
Registered: October 2004
Location: Third star to the left
Aeglos wrote on Thu, 07 July 2011 13:22

Bots as in AI?

We have code in place that runs AI already, it pretty basic for now; build a couple of things and sends scouts. I don't know what's the plan for the AIs as I haven't touched that yet, but custom AIs can be made, albeit in C#.

Perhaps 3rd parties could write decent AIs (in whatever platform they chose) and relieve the Nova team from that burden. Deal



So many Stars, so few Missiles!

In space no one can hear you scheme! Deal

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 07:41 Go to previous messageGo to next message
paul_ik is currently offline paul_ik

 
Master Chief Petty Officer

Messages: 98
Registered: November 2009
Location: Belarus GMT+2

Aeglos wrote on Thu, 07 July 2011 14:22


Bots as in AI?

We have code in place that runs AI already, it pretty basic for now; build a couple of things and sends scouts. I don't know what's the plan for the AIs as I haven't touched that yet, but custom AIs can be made, albeit in C#.



Yes bots as in AI. Smile

I've seen AI code in your project, and while it is a very important thing to have default AI developed as a part of the project, I see this problem as an interesting challenge for third party developers.

I don't program in C#, that's why I want to use turn files. Besides it will have a nice testing effect for your project.

Once you have turn files format mostly fixed we can write AI with different styles which is a great fun Smile
We can even organize a kind of contest for AI developers, but I'm going too far here Smile
I hope you get my point.

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 07:54 Go to previous messageGo to next message
m.a@stars is currently offline m.a@stars

 
Commander

Messages: 2760
Registered: October 2004
Location: Third star to the left
paul_ik wrote on Thu, 07 July 2011 13:25

CSV is convenient if you have flat data, containing one-to-one relations, not the thing you will expect with stars.

Not?

Deal player relations (as an NxN grid)
Deal planet stats (original and current)
Deal production Qs (if you treat them like strings of orders)
Deal ship designs (if you treat them like strings of components)
Deal fleet data (including their strings of orders)
Deal battle reports (if you treat them like strings of events)
Deal tech levels (including one 'status' column with values such as 'current' and 'next')
Deal item, engine and hull definitions
Deal scores history
Deal race definitions (if you treat them like strings of options)
Deal initial techs and ships
Deal perhaps even turn orders too.

In fact, I'd dare say current Stars! has most everything as flat arrays with a few relationships (not necessarily one-to-one) between them. Deal


Quote:

XML is more flexible - you can add elements here and there, perfect feature for an actively developed project.

Indeed. Nod But it can also be overkill.

Quote:

So among textual formats I think XML is pretty nice choice.

It's not a bad choice. But it might not be the only choice. Deal



So many Stars, so few Missiles!

In space no one can hear you scheme! Deal

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 07:57 Go to previous messageGo to next message
m.a@stars is currently offline m.a@stars

 
Commander

Messages: 2760
Registered: October 2004
Location: Third star to the left
paul_ik wrote on Thu, 07 July 2011 13:41

Yes bots as in AI. Smile

I've seen AI code in your project, and while it is a very important thing to have default AI developed as a part of the project, I see this problem as an interesting challenge for third party developers.

I don't program in C#, that's why I want to use turn files. Besides it will have a nice testing effect for your project.

Once you have turn files format mostly fixed we can write AI with different styles which is a great fun Smile
We can even organize a kind of contest for AI developers, but I'm going too far here Smile

Yes Dueling Pirate Cheers Thumbs Up Very Happy Deal New idea Twisted Evil



So many Stars, so few Missiles!

In space no one can hear you scheme! Deal

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 08:02 Go to previous messageGo to next message
Aeglos is currently offline Aeglos

 
Chief Warrant Officer 1
Stars! Nova developer
Stars! Nova developer

Messages: 142
Registered: May 2011
Location: Chile
There is one issue though; to make bots from the turn files you would need to make a whole turn generator, and you would need all of the player files, not just one.

Since the .intel files are complementary, you can't extract the whole game state from just one of them.

The ideal situation would be for us to make some king of pluggable AI through LUA or some other language, but that is pretty low on our priorities... i'd dare say near zero right now.

I guess we'll see what you guys devise with the turn files once their specs are finished Twisted Evil

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 08:14 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
Trying to read all the above..

File Format:
If it were flat tables then I'd be more inclinded to use an embedded database like SQLite which is awesome for stuff like this and does all the storing/parsing for you. And indeed I may well have designed it that way from the outset.

However we're working on a code base we didn't start and that's not how it's been designed, so given that XML is a very reasonable way to go.



AI Bots:
If you want to write an AI (and please, please do) then I would suggest you write it in the actual Nova code, rather than as a separate process.

Ideally there would be a plugin system for this so people could write AIs against an API in other languages but that's not on any priority list yet.

@paul_ik - what do you program in now?


[Updated on: Thu, 07 July 2011 08:15]

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 09:11 Go to previous messageGo to next message
paul_ik is currently offline paul_ik

 
Master Chief Petty Officer

Messages: 98
Registered: November 2009
Location: Belarus GMT+2

Aeglos wrote on Thu, 07 July 2011 15:02

There is one issue though; to make bots from the turn files you would need to make a whole turn generator, and you would need all of the player files, not just one.

Since the .intel files are complementary, you can't extract the whole game state from just one of them.

The ideal situation would be for us to make some king of pluggable AI through LUA or some other language, but that is pretty low on our priorities... i'd dare say near zero right now.

I guess we'll see what you guys devise with the turn files once their specs are finished Twisted Evil


I don't catch - why do I need turn generator implementation?
The bot will behave as a regular player - taking static data, history and latest turn as input and returning orders.

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 09:25 Go to previous messageGo to next message
paul_ik is currently offline paul_ik

 
Master Chief Petty Officer

Messages: 98
Registered: November 2009
Location: Belarus GMT+2

Musmuris wrote on Thu, 07 July 2011 15:14



AI Bots:
If you want to write an AI (and please, please do) then I would suggest you write it in the actual Nova code, rather than as a separate process.

Ideally there would be a plugin system for this so people could write AIs against an API in other languages but that's not on any priority list yet.

@paul_ik - what do you program in now?


Why inventing plugin system when we have files ? Why programming against API when we have formats documentation ?

Am I missing something? Here is my understanding.

Server (written in C#) generates turns. No problem with that. I'm not talking about hosting games anyway Razz

Clients are used to work with turns, they can be written in any language, using any tools. They take input files and return order file. AI is a client.

I want to use python for this.

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 09:38 Go to previous messageGo to next message
m.a@stars is currently offline m.a@stars

 
Commander

Messages: 2760
Registered: October 2004
Location: Third star to the left
Aeglos wrote on Thu, 07 July 2011 14:02

to make bots from the turn files you would need to make a whole turn generator

You mean a player-side turn crafter, right? Sherlock


Quote:

you would need all of the player files, not just one.

As I imagine it, any player, AI or not, will only have its own files to work with, plus perhaps some historical records and previous analyses. Deal

That would after all be one of the main features of a client-server architecture, wouldn't it?



So many Stars, so few Missiles!

In space no one can hear you scheme! Deal

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 09:39 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
Mainly because file formats can and frequently do change. If it's an API then tests can be written against it and the API abstracts the changes. If it were me that's what I'd prefer to do. Plus it saves re-creating all the object model.

However What you're suggesting works fine and I certainly wouldn't want to stop you doing that. It's just that having the ability to take a (e.g.) python AI and make that part of the distribution for everyone running a server has an appeal. (and yes obviously that can be done without a plugin system as well - it just makes more sense to me that as it's open source anyway makeing it all one codebase seems more logical to me)

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 09:43 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
Just to follow up - yes having a standalone python client is a great idea.

I genuinely thought you might prefer a way to plugin (e.g.) python the server as an AI but either way is cool.

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 09:45 Go to previous messageGo to next message
Daniel is currently offline Daniel

 
Chief Warrant Officer 3
Stars! Nova developer
Stars! Nova developer

Messages: 179
Registered: April 2006
Location: Nowra, Australia
Why XML? Because I found the binary serialized data impossible to debug and I was interested in learning about XML. Rolling Eyes I could have written it as a flat file format or used a database but interest in learning a different approach was my driver for doing the work. Laughing The advantage of XML is that it is very forgiving of layout and order for 3rd party tools, but that was not the reason it was chosen. Very Happy

As far as documenting the format goes - don't hold your breath. The files are gziped XML and are fairly human readable (after uncompressing). For further documentation you will have to read the code or look at the default AI. Work at computer My expectation is that once the file formats are fairly stable (I expect by version 0.6 they should be), then someone interested in developing 3rd party stuff may do some documentation of them. As we are at 0.4 and 0.5 has been more than 12 months in the works you have a bit of time to wait, unless you want to code AIs in C# using Nova .dlls.



Have fun.

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 09:47 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
To follow up again!

Have you thought about learning C#? Twisted Evil You'd be really welcome to come and write an AI actually in the codebase directly.

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 09:50 Go to previous messageGo to next message
m.a@stars is currently offline m.a@stars

 
Commander

Messages: 2760
Registered: October 2004
Location: Third star to the left
Musmuris wrote on Thu, 07 July 2011 14:14

we're working on a code base we didn't start and that's not how it's been designed, so given that XML is a very reasonable way to go.

You're building a client-server system, aren't you? Then having some choice in intermediate data-exchanging formats makes sense, and would indeed help with the creation of additional clients, or even servers. Deal


Quote:

AI Bots:
If you want to write an AI (and please, please do) then I would suggest you write it in the actual Nova code, rather than as a separate process.

Ideally there would be a plugin system for this so people could write AIs against an API in other languages

I guess what you mean is that any AI bot/client would greatly benefit from having access to the things the current Nova client (or server) can do, starting with all the maths, the processing of all the game's data, and everything. Deal

Perhaps a short-term fix could be to allow external processes to call or launch Nova (either the client or the server) with some kind of special parameters to get certain sets of results. Sherlock



So many Stars, so few Missiles!

In space no one can hear you scheme! Deal

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 09:50 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
Daniel wrote on Thu, 07 July 2011 09:45

As we are at 0.4 and 0.5 has been more than 12 months in the works


Though currently there seems more momentum so hopefully things will happen faster (though obviously no promises)

Report message to a moderator

Re: Turn files documentation Thu, 07 July 2011 09:52 Go to previous messageGo to previous message
m.a@stars is currently offline m.a@stars

 
Commander

Messages: 2760
Registered: October 2004
Location: Third star to the left
Musmuris wrote on Thu, 07 July 2011 15:39

file formats can and frequently do change.

That happens with databases too. There's ways to cope with that. Plus, public access to the data files helps keep everyone honest. Deal



So many Stars, so few Missiles!

In space no one can hear you scheme! Deal

Report message to a moderator

Previous Topic: Stars! Nova May 2011
Next Topic: Starting a Masters
Goto Forum:
  


Current Time: Wed Jul 28 06:54:51 EDT 2021