Home World Forum
Stars! AutoHost web forums

Jump to Stars! AutoHost


 
 
Home » Stars! Clones, Extensions, Modding » FreeStars » A modest proposal to make clients easier
A modest proposal to make clients easier Sun, 29 January 2006 07:34 Go to next message
Madman is currently offline Madman

 
Officer Cadet 1st Year

Messages: 228
Registered: November 2003
Location: New Zealand
[Update - some test results for this idea later in the thread http://starsautohost.org/sahforum/index.php?t=msg&th=278 0&start=36 ]

One of the bottlenecks with this project is a lack of a client.

Looking at the clients that have been/are being worked on, there are 3 C++ client attempts (one of which is part of the Freestars project), one in Visual Basic, one in Delphi (although the author of that packed up his toys and went home so we can discount that one), with proposals to write clients in Java or Python.

All of these (with the exception of the one in FreeStars) suffer from a particular problem - that it is a maintainence problem to keep the game mechanics of the client in sync with that of the server (for instance, the client giving freedback on fuel usage, or if the client wants to have a battle simulator).

Those projects using C++ do have the advantage that that can grab the code from the server, but this still requires maintainence, particularly if anyone ever wants to write a customized server (they'd have to change the client to match).

My proposal is that an instance of the server be run on the client, so it can be used to deal with all game mechanics, with queries and replies written in XML to keep things string based.

The setup would be like this (view in a fixed width font):

Client machine Server machine
+------+XML+---------------------+ +-------------------+
|Client|<=>|Client copy of server| |Server running game|
+------+ +---------------------+ +-------------------+

The client copy of the server (called the local server for short) would not have any information that player wasn't supposed to have, so there would be no danger of information leaking that isn't supposed to be.

All communication between the client and the local server would be ascii string based (using XML) so there aren't any language data structure or unicode problems. Communication cound be done by making the local server a shared library, or sockets or possibly pipes.

For example, say I want to send a fleet from one point to another in the client, and want instant feedback on fuel usage.

The client would send a message something like (the syntax might be qute different than this):

<move_query from_x='324' from_y='234' to_x='388' to_y='211' fleet_id='23'/>

where the local server already knows the race of the player, fleet composition, cargo etc.

The reply might be something like:
<move_reply>
<speed warp='1' time='64' fuel='0'/>
<speed warp='2' time='16' fuel='0'/>
<speed warp='3' time='8' fuel='0'/>
<speed warp='4' time='4' fuel='0'/>
<speed warp='5' time='3' fuel='3'/>
<speed warp='6' time='2' fuel='10'/>
<speed warp='7' time='2' fuel='20'/>
<speed warp='8' time='1' fuel='40'/>
<speed warp='9' time='1' fuel='50'/>
<speed warp='10' time='1' fuel='60' overload_risk='t10%'/>
<speed warp='gate' time='1' fuel='0' success='uncertain'/>
</move_reply>

There would need to be messages for anything that the client can display that is not included directly in the turn file, such as what is going to happen to a planet next year, the mineral and resource cost of ship designs, and a rather more complex message for sending fleet compositions and getting battle results if the client was to have a battle simulator.

If at some point there is a customized server, as long as it returns the messages correctly, the client wouldnt even need to know (there might have to be a version message to check that the the local server is the same version as the one running the game).

I suggest that having a documented 'language' like this would make clients much simpler to write and maintain. I am willing to do the work to get this included in the server.

Questions:
=========
(1) Do the maintainers of the project think this would be a useful direction to go? Would they accept this going into the project?
(2) Of those people working on clients or considering working on clients, would you make use of such an interface if provided?

I'd be interested to read your thoughts before I do anything with this.


[Updated on: Mon, 06 February 2006 16:11]

Report message to a moderator

Re: A modest proposal to make clients easier Sun, 29 January 2006 12:29 Go to previous messageGo to next message
Kotk

 
Commander

Messages: 1227
Registered: May 2003
So you want to make the game generator/turn generator server to do some client job? Or you want to make second type of server that actually does client job? Very Happy

Usually it seems wise to split the tasks between modules instead of burdening lot of functionality onto one module. For example to have Game Generator as module (makes .XY .HST and initial .M# files), Turn Generator as other module, Battle Engine as turn generators submodule and so on. Calling a function from a module is simpler than making modules to talk XML with each other. Rolling Eyes

Better idea is to document the current server and its classes and functions bit better and then reuse them or call them. Its usually better if something is missing but documented as to be done in contrast with it being implemented but nowhere mentioned. Confused

Making and documenting new XML interfaces instead of documenting and calling the functions themselves ... i suspect it does speed nothing but adds new stuff to be documented and implemented? Nod

Since i am writing some AI i answer only to Q2 ... i sure embed classes as code (or dynamic load libraries if these are made) from some Server or UI version and call them when i get so far. I think its more complex to make libraries that talk XML with other libraries in same computer.

Report message to a moderator

Re: A modest proposal to make clients easier Sun, 29 January 2006 16:39 Go to previous messageGo to next message
Madman is currently offline Madman

 
Officer Cadet 1st Year

Messages: 228
Registered: November 2003
Location: New Zealand
Kotk wrote on Mon, 30 January 2006 06:29

So you want to make the game generator/turn generator server to do some client job? Or you want to make second type of server that actually does client job? Very Happy

Neither.

The instance of the server running on the game never does any client jobs - there would be all sorts of problems with that, such as not being able to do stuff offline.

I'm also not suggesting a second type of server - it would actually be a copy of the same server running on the user's machine - it is _very_ important that the game mechanics code need only be written once - but only a limited part of it would be used. Actuaully there could be build options so that the client talking code is not built into the copy of the server running the game, and the turn generation stuff is not built into the client copy, but that doesn't really achieve anything apart from some small memory savings.

Quote:

Usually it seems wise to split the tasks between modules instead of burdening lot of functionality onto one module. For example to have Game Generator as module (makes .XY .HST and initial .M# files), Turn Generator as other module, Battle Engine as turn generators submodule and so on. Calling a function from a module is simpler than making modules to talk XML with each other. Rolling Eyes

Better idea is to document the current server and its classes and functions bit better and then reuse them or call them. Its usually better if something is missing but documented as to be done in contrast with it being implemented but nowhere mentioned. Confused

Making and documenting new XML interfaces instead of documenting and calling the functions themselves ... i suspect it does speed nothing but adds new stuff to be documented and implemented? Nod


Well firstly, I agree that creating and documenting an interface (maybe such an interface would best be done in C rather than C++ unless we want to absolutely lock in C++ as the only caller) of 'these are the only functions that the client needs to call' would be a good idea, as it would be useful in some cases. A C/C++ client can link the server or the appropriate part of the server in as a library, and make those calls directly.

However, that locks any clients in to being written in C/C++, and my thesis is that that limitation is part of why we don't have a client yet.

For instance, if I want to write a Java client (I'm fast at Java, and already know the UI classes in that), with only direct calls available, I'm faced with two options:
(1) Have the Java code linked to a C++ library using JNI or similar - this tends to be very error prone and really limits the complexity of the calls - there will never be a battle simulator in the client for instance.
(2) Reimplement the game mechanincs in Java. This is poor design for several reasons I mentioned in the 'Compiling on Linux' thread. It also breaks each time there is a game mechanics change in the server, such as desiging a varient version.

I use the XML message approach in my paid work (getting PHP to talk to a Java server) and it works very well (and is great for debugging) - although in that case it doesn't need to be portable.

For me, writing a Java client _and_ the messaging scheme would be actually be quicker than writing a client in C++, given that for the first case I already know all the technology I need to use.

Report message to a moderator

Re: A modest proposal to make clients easier Sun, 29 January 2006 17:12 Go to previous messageGo to next message
LEit is currently offline LEit

 
Lt. Commander

Messages: 879
Registered: April 2003
Location: CT
The server is compiled as a static library now, That should be callable from most other languages, however, they deal with classes a lot, and I'm not sure how well those will work with other langauges.

XML would be a way to deal with that, but it would also add a lot of overhead (coding and maintenance of that code, size of exe I don't worry about). If a client developer tells me that's what they need, I'll consider it I suppose.



- LEit

Report message to a moderator

Re: A modest proposal to make clients easier Sun, 29 January 2006 17:31 Go to previous messageGo to next message
Madman is currently offline Madman

 
Officer Cadet 1st Year

Messages: 228
Registered: November 2003
Location: New Zealand
LEit wrote on Mon, 30 January 2006 11:12

The server is compiled as a static library now, That should be callable from most other languages, however, they deal with classes a lot, and I'm not sure how well those will work with other langauges.

I can give you a simple answer: classes won't work with other languages, or at least any of teh ones I've looked at. C++ uses name mangling in it's libraries, and how that is done will vary from complier to complier, and even between compiler versions (which is part of why you code written with different versions of gcc may not be able to be linked together).

This is whay a lot of libraries will have

extern "C"

in front of or around a lot of it's declarations.

LEit wrote on Mon, 30 January 2006 11:12

XML would be a way to deal with that, but it would also add a lot of overhead (coding and maintenance of that code, size of exe I don't worry about). If a client developer tells me that's what they need, I'll consider it I suppose.

Well, if there's any interest, I'll do a small test case, such as for instance writing a graphical race wizard in Java that uses the C++ machinery.

Report message to a moderator

Re: A modest proposal to make clients easier Sun, 29 January 2006 19:44 Go to previous messageGo to next message
Madman is currently offline Madman

 
Officer Cadet 1st Year

Messages: 228
Registered: November 2003
Location: New Zealand
For completeness, I should have mentioned that there is one protocol that does allow object communcation between different languages - something called CORBA (Common Request Broker Architecture).

I have experience with that, but it is an extremely heavyweight architecture, and I don't recommend it for this project.

COM or DCOM or OLE or ActiveX (or whatever they are calling it nowadays) might also do the same thing for Windows (I don't really know much about it), but it won't work on any other platform.

Report message to a moderator

Re: A modest proposal to make clients easier Wed, 01 February 2006 13:33 Go to previous messageGo to next message
Kotk

 
Commander

Messages: 1227
Registered: May 2003
Madman wrote on Mon, 30 January 2006 00:31

Well, if there's any interest, I'll do a small test case, such as for instance writing a graphical race wizard in Java that uses the C++ machinery.
What C++ machinery? The races are XML already?
I think that ... a client that eats, displays and outputs already existing XML files would be cool enough for starters. Wink

Report message to a moderator

Re: A modest proposal to make clients easier Wed, 01 February 2006 15:29 Go to previous messageGo to next message
Madman is currently offline Madman

 
Officer Cadet 1st Year

Messages: 228
Registered: November 2003
Location: New Zealand
Kotk wrote on Thu, 02 February 2006 07:33

What C++ machinery? The races are XML already?
I think that ... a client that eats, displays and outputs already existing XML files would be cool enough for starters. Wink


Well, that would certainly do for a start, but eventally in the client you want to be able to do stuff that is more than just reading XML, such as sending a ship somewhere to see how much fuel it is going to take, or look at what population a planet will have next turn, or simulate a battle. That's the C++ machinery that I don't think should be duplicated in the client, hence the message idea (for non-C++ clients).

The example of the Race wizard is just to calculate race wizard points - the server has got to know how to do this anyway, so much better having the server code do it than duplicate the code in the client.

I'm currently having a look at C++ clients, so I'm only going to provide this if I decide to go write a Java one anyway, or if someone else wanting to write a non-C++ client would use it, as there's no point writing code that isn't going to be useful to anyone. No takers so far.


[Updated on: Wed, 01 February 2006 16:11]

Report message to a moderator

Re: A modest proposal to make clients easier Wed, 01 February 2006 20:30 Go to previous messageGo to next message
Kotk

 
Commander

Messages: 1227
Registered: May 2003
Madman wrote on Wed, 01 February 2006 22:29

The example of the Race wizard is just to calculate race wizard points - the server has got to know how to do this anyway, so much better having the server code do it than duplicate the code in the client.

I understood there is rule xml file that among other things contains RW rules. Quite schematic last i checked, but idea was to have point costs for things in RW specified there.

Lets imagine now the situation:
Client wants to know how lot a race has +-rw points.
Client feeds server with race, with the rules for what the race is built (probably ~ half megabyte file) and question about rw points.
Server eats it all, ponders and replies to UI that -41 rw points.
UI displays that number.
User clicks some hab one click left and the whole process repeats? Confused

Report message to a moderator

Re: A modest proposal to make clients easier Wed, 01 February 2006 22:58 Go to previous messageGo to next message
Madman is currently offline Madman

 
Officer Cadet 1st Year

Messages: 228
Registered: November 2003
Location: New Zealand
Kotk wrote on Thu, 02 February 2006 14:30


I understood there is rule xml file that among other things contains RW rules. Quite schematic last i checked, but idea was to have point costs for things in RW specified there.

Lets imagine now the situation:
Client wants to know how lot a race has +-rw points.
Client feeds server with race, with the rules for what the race is built (probably ~ half megabyte file) and question about rw points.
Server eats it all, ponders and replies to UI that -41 rw points.
UI displays that number.
User clicks some hab one click left and the whole process repeats? Confused

Well, the local server only needs to load the rules once on startup - it can remember it after that. Apart from that, yes, you've got it right.

A race file is about 2.5 KB, so the local server (remember it's not the server running the game, but something local, so communication will be quick) should easily be able to respond fast enough.

Actually, testing that it's got a good response time is part of the reason for doing RW as the first test - because changing one click somewhere needs communication with the points calculation code, it is likely to be the most interactive of all the tasks that this interface would be used for. If it can handle a RW OK, then it is very likely to be fast enough for the other tasks.

The alternative is to have a whole lot of logic for working out RW points (remember there are all sorts of interactions between the race parameters which are unlikely at this stage to all be encoded in the rules file) inside the client.

Report message to a moderator

Re: A modest proposal to make clients easier Thu, 02 February 2006 00:13 Go to previous messageGo to next message
Kotk

 
Commander

Messages: 1227
Registered: May 2003
Okay ... so we assume then that people do not play in games with different rules at same time. If they do they do not open multiple clients with different rule games at same time?

Even fuel consumption sounds nothing simple. Asking about fuel consumption the client has to provide race design (ife?), ship designs, exact fleet composition, cargo and waypoints. Server has to be ready anytime to read all that stuff, make calculations and provide answers to client that asked it. I have usually my teamie turn also open plus often a testbed so server cant assume that it is same race or game from previous query.

Maybe each open client runs also local personal server so server does not need to reload stuff each time when some client asks it? Then it all sounds like statically linked library that discusses things in xml with its main module? Very Happy

Overall it ... feels that C++ is not so hard to learn for client maker ... at least lot easier thing than to write all that xml communication crap for every little click that user makes. Wink

Report message to a moderator

Re: A modest proposal to make clients easier Thu, 02 February 2006 00:54 Go to previous messageGo to next message
Madman is currently offline Madman

 
Officer Cadet 1st Year

Messages: 228
Registered: November 2003
Location: New Zealand
Kotk wrote on Thu, 02 February 2006 18:13

Okay ... so we assume then that people do not play in games with different rules at same time. If they do they do not open multiple clients with different rule games at same time?

No problem - each client would have it's own''local server' bundled in, so multiple clients -> multiple servers. In fact it would preferably be arranged so that this server opening was invisible to the person running the client.

Quote:

Even fuel consumption sounds nothing simple. Asking about fuel consumption the client has to provide race design (ife?), ship designs, exact fleet composition, cargo and waypoints. Server has to be ready anytime to read all that stuff, make calculations and provide answers to client that asked it. I have usually my teamie turn also open plus often a testbed so server cant assume that it is same race or game from previous query.

The local server associated with each client would be told and remeber race, ship designs, etc. so none of this is a problem.

Quote:

Maybe each open client runs also local personal server so server does not need to reload stuff each time when some client asks it? Then it all sounds like statically linked library that discusses things in xml with its main module? Very Happy

Now you've got it Smile In many cases it would be a statically linked module. What the XML stuff gives is a way of passing complicated information between different languages.

Quote:

Overall it ... feels that C++ is not so hard to learn for client maker ... at least lot easier thing than to write all that xml communication crap for every little click that user makes. Wink


Actually, the 'xml communication crap' as you call it would not need to be called for every click (well, it would in the race wizard, but not elsewhere), just when the client needs to get information it doesn't have on hand - I think fuel use would come up the most often. I've used something like this approach with some success in two other projects.

As for "C++ is not so hard to learn for client maker" - it's not all that hard to learn C++ (pig of a language that it is, I already know it and have programmed in it professionally). The problem is learning an GUI toolkit for it. For instance, if I had time to take on the whole client myself, it would be quicker for me to add the communication layer in xml to the server _and_ write a Java client than get something working in C++, and have to learn a GUI toolkit I'm going to use for _one_ project.

There has been interest expressed in non-C++ clients - all I'm proposing is a way of enabling that without duplicating a lot of logic. If someone wants to write a C++ client, then by all means use the classes directly, and my scheme becomes unecessary for that.

I'm also open to other suggestions to making clients in other languages feasible - that's part of why I put this idea out there.

Report message to a moderator

Re: A modest proposal to make clients easier Thu, 02 February 2006 13:57 Go to previous messageGo to next message
Kotk

 
Commander

Messages: 1227
Registered: May 2003
Madman wrote on Thu, 02 February 2006 07:54

I'm also open to other suggestions to making clients in other languages feasible - that's part of why I put this idea out there.

But there is JNI for making layer between C++ and Java if needed. Why not to use it? C interface layer is there for most languages. Smile Usually does not take rocket science to use. Confused If one cant use native interface then he probably fails with some xml layer even worse. Nod

Report message to a moderator

Re: A modest proposal to make clients easier Thu, 02 February 2006 15:21 Go to previous messageGo to next message
Madman is currently offline Madman

 
Officer Cadet 1st Year

Messages: 228
Registered: November 2003
Location: New Zealand
Kotk wrote on Fri, 03 February 2006 07:57

But there is JNI for making layer between C++ and Java if needed. Why not to use it? C interface layer is there for most languages. Smile Usually does not take rocket science to use. Confused If one cant use native interface then he probably fails with some xml layer even worse. Nod

I wrote about this in a post on the 'Compiling on Linux' thread. Basically, the issue is that while JNI allows connection to java (and other languages have similar mechanisms), it doesn't really lend itself to passing data structures between the languages. In particular, you can't use the C++ classes (unless there's been some massive improvement to JNI of which I'm not aware - something like gcj _might_ work for Java). Someone else who has used JNI please come on and tell me I'm wrong.

All the XML does (and it's not that complicated to do) is marshall and unmarshall the data into a string. I think a lot of the objections to this are based on the perception that it is somehow really complex when actually it's not that bad.

Once the call to the game mechanics code is all of the form:

xml_string query(xml_string);

or possibly

xml_string query(query_name_string, xml_string);

then the communication method becomes easy, whether it is something like JNI, sockets (I'd use sockets while developing) or pipes.

As a small side effect, in the other project where I used this, debugging and testing becomes much easier too - just log the XML messages for debugging, or have a test program send XML messages for automated testing.

Report message to a moderator

Re: A modest proposal to make clients easier Fri, 03 February 2006 06:41 Go to previous messageGo to next message
Kotk

 
Commander

Messages: 1227
Registered: May 2003
Madman wrote on Thu, 02 February 2006 22:21

Basically, the issue is that while JNI allows connection to java (and other languages have similar mechanisms), it doesn't really lend itself to passing data structures between the languages. In particular, you can't use the C++ classes (unless there's been some massive improvement to JNI of which I'm not aware - something like gcj _might_ work for Java).

Are you sure that half of the server library doesnot need to be rewritten to support these xml queries? Yes, JNI interface functions have to use "C" calling convention. How can such things be compared? If you say that exchanging files and xml is not that bad, slow or complex to use as a communication medium then using C calls and passing C structures sound simply revolutionarily brilliant. Laughing wOOt 2
Bottom line is that do what you want if its fun for you to do. Even if it later appears to be too slow for some things then for some things such MSQI (madman stars query interface) is probably good enough. Nod


[Updated on: Fri, 03 February 2006 07:21]

Report message to a moderator

Re: A modest proposal to make clients easier Fri, 03 February 2006 08:50 Go to previous messageGo to next message
Madman is currently offline Madman

 
Officer Cadet 1st Year

Messages: 228
Registered: November 2003
Location: New Zealand
Kotk wrote on Sat, 04 February 2006 00:41

Are you sure that half of the server library doesnot need to be rewritten to support these xml queries?

Yes, I'm absolutely sure - it's just a layer on top, only a little more work that what you'd have to do for a C interface, and something I know how to do because I've done something very similar before. Also, nothing I would do would prevent anyone from using another interface.

There's actually only about 5 things I've thought of so far that can't just be pulled out of the turn file:
* race wizard points,
* calculate fuel usage and travel times,
* ship design - either implement as returning the cost of components for ship and starbase and add them in the client (making sure the rounding is the same), or have the local server take a ship design and return total costs,
* extended planet information - such as hab percentage, expected growth, how many resources will be available, how many minerals will be generated, how much terraforming can be done,
* battle simulator (which isn't even in the current Stars! program but I would really like to have).
There are probably other things that make the whole thing easier, particularly the client updating the local server with things like changed population/minerals on planets, new ship designs and fleet composition changes, but these would be a good start.

Quote:

Yes, JNI interface functions have to use "C" calling convention. How can such things be compared? If you say that exchanging files and xml is not that bad, slow or complex to use as a communication medium then using C calls and passing C structures sound simply revolutionarily brilliant. Laughing

Well, to start with, I've nowhere talked about 'exchanging files' so you can ditch that straw man now. Exchanging files is slow and deadlock prone (I tried it once, then switched to sockets which worked great).

I've interfaced garbage collected langauges with C before, and found that it was painful and prone to have memory management issues, and that the C interface is really only any good for passing simple stuff such as returning a simple number - incidentally, it would probably work fine for the RW queries where a single integer is returned, just not for much else, certainly not for something like a battle simulator.
Quote:

Bottom line is that do what you want if its fun for you to do.

Of course - all open source development works that way.
Quote:

Even if it later appears to be too slow for some things then for some things such MSQI (madman stars query interface) is probably good enough. Nod

Not a bad name Smile Although MSQL (madman stars query language) is way more buzzword compliant - on second thoughts, that sounds too much like a database, so MSQI it is for the moment Smile
I very much doubt it would be too slow if written properly - I've got a similar interface where it is passing hundreds of such messages per second - we are talking round about millisecond times here (unless you want to run it on something like a Pentuim 75, in which case non C/C++ clients are going to be too slow anyway), and I don't know about you, but I just can't click that fast Laughing
And as I've mentioned a few times, I'm only going to do it if it's useful for some client that someone actually wants to write. Most of the suggestions I have seen involving duplicating the game logic in the client, and that's a long term maintainence headache I'd really like to avoid.

Report message to a moderator

Re: A modest proposal to make clients easier Fri, 03 February 2006 10:02 Go to previous messageGo to next message
m.a@stars is currently offline m.a@stars

 
Commander

Messages: 2765
Registered: October 2004
Location: Third star to the left
Madman wrote on Sun, 29 January 2006 13:34

All of these (with the exception of the one in FreeStars) suffer from a particular problem - that it is a maintainence problem to keep the game mechanics of the client in sync with that of the server (for instance, the client giving freedback on fuel usage, or if the client wants to have a battle simulator).

Those projects using C++ do have the advantage that that can grab the code from the server, but this still requires maintainence, particularly if anyone ever wants to write a customized server (they'd have to change the client to match).



Hi,

Pls allow me to intrude, gentlemen, as I've often pondered on these matters myself:

If we think about a Stars client that will be almost like the original one, it will obviously need to display some things in real-time which will require some math, such as fuel usage (and ship range), ship rating, RW points, habitability values (and expected growth), Research budgets (and costs), mining ratios, scanner ranges...

But these are not overly costly or even complex calculations, and the worst of them (RW and Hab) are more related than you may believe. So, I think at least part of the Server should indeed be "replicated" or bundled with the Client. But it will be a very simple part, or at least one part which should be pretty fixed, such as the RW and Hab calculations, where only a certain amount of ruleset or game-creation tweaks can be allowed.

The Battle Simulator seems to me a very different matter. The actual Battle Engine doesn't need to know about display at all, so long as it can read a fleet configuration and output a logfile (or bit-packed stream or whatever) which a graphic client can render into a battle VCR, or the turn-generating server can use to churn along. So, it could become a closed box, callable from all parties, or even a stand-alone program (which might be easier to do than having it running under any kind of middleware).

I'd say the better thing would be setting in place *any* kind of graphic Client at all, able to communicate with the current Server to accomplish what simple tasks can currently be accomplished, and then work on the best way to finish linking them both. Best practice here should be less important than say, getting a beta-testing community up and running before Stars itself goes the way of the Dodo. :-/

Please someone craft a nice, appealing Client and set things rolling and force everybody else (me included) to follow along, yes? Very Happy


[Updated on: Fri, 03 February 2006 10:03]




So many Stars, so few Missiles!

In space no one can hear you scheme! Deal

Report message to a moderator

Re: A modest proposal to make clients easier Fri, 03 February 2006 11:18 Go to previous messageGo to next message
Kotk

 
Commander

Messages: 1227
Registered: May 2003
Madman wrote on Fri, 03 February 2006 15:50

There's actually only about 5 things I've thought of so far that can't just be pulled out of the turn file:
* race wizard points,
* calculate fuel usage and travel times,
* ship design - either implement as returning the cost of components for ship and starbase and add them in the client (making sure the rounding is the same), or have the local server take a ship design and return total costs,
* extended planet information - such as hab percentage, expected growth, how many resources will be available, how many minerals will be generated, how much terraforming can be done,
* battle simulator (which isn't even in the current Stars! program but I would really like to have).

When i look at current client then i see few more:
* prod que: 1 year prognose (greens and blues) 100 years prognose (blues, blacks and reds)
* fleet: cloaking, gate jump prognose (its too simple) and estimated travel range (its also too simple) also fleet shield and armor would be fine to see.
* minefield: decay rate
* research: next year prognose
Probably some more ... yes its nice idea to have full list before doing something.
Tools like habitability (with TT CA ally you have far better hab than current client show), opponents scan prognose, growth, battle, packet or bombing simulators ... these things make client lot better.
Quote:

I've interfaced garbage collected langauges with C before, and found that it was painful and prone to have memory management issues, and that the C interface is really only any good for passing simple stuff such as returning a simple number - incidentally, it would probably work fine for the RW queries where a single integer is returned, just not for much else, certainly not for something like a battle simulator.

My experience show that it is less bad than you say here. For JNI, C++ side has to manage memory for the stuff that it passes. It does not need to be C, just the interface is with C calls. Yes, the battles take lot of room in turn files. For current Stars! it passes too few data so there are battle display flaws and you can even crash the game engine by making too big battles. 100 fleets living 16 rounds each doing 3 phases per round (sounds we talk about megabyte of xml here) so memory management must be flawless on any case no difference if you pass arrays of characters (xml) or arrays of structures.
Quote:

And as I've mentioned a few times, I'm only going to do it if it's useful for some client that someone actually wants to write. Most of the suggestions I have seen involving duplicating the game logic in the client, and that's a long term maintainence headache I'd really like to avoid.

I fully agree that some interface must be there if people want to make non-C++ clients. You say have seen evidence that xml is quick and so i got to trust you. My suspision remains, but it does never hurt when making software. Wink I am sure that even C++ UI is not so easy to write and learning to use some crossplatform vidget GUI in the process is smallest of the problems. Very Happy Because no gui developers have replied i have nothing to do but discuss it to keep the topic up. Very Happy

Report message to a moderator

Re: A modest proposal to make clients easier Fri, 03 February 2006 16:27 Go to previous messageGo to next message
Madman is currently offline Madman

 
Officer Cadet 1st Year

Messages: 228
Registered: November 2003
Location: New Zealand
m.a@stars wrote on Sat, 04 February 2006 04:02

I'd say the better thing would be setting in place *any* kind of graphic Client at all, able to communicate with the current Server to accomplish what simple tasks can currently be accomplished, and then work on the best way to finish linking them both. Best practice here should be less important than say, getting a beta-testing community up and running before Stars itself goes the way of the Dodo. :-/

OK, I agree here - I don't intend my idea to get in the way of someone writing a client in whatever way gets it out the door - what this proposal was _meant_ to do was say "if someone wants to write a client in whatever language they like, I'm prepared to offer a mechanism to get the server to to all the game mechanics".

Even something that just lets you work with the stuff in the turn files would be a good start!

Quote:

But these are not overly costly or even complex calculations, and the worst of them (RW and Hab) are more related than you may believe. So, I think at least part of the Server should indeed be "replicated" or bundled with the Client. But it will be a very simple part, or at least one part which should be pretty fixed, such as the RW and Hab calculations, where only a certain amount of ruleset or game-creation tweaks can be allowed.

There are several issues I see with this:
* There are some subtleties (such as rounding) that do make this code non-trivial. Even the current Stars! occasionally gets it wrong for things like travel distances - do you really think that something travelling 98.97 light years at warp 7 is always going to get there in 2 years? I suspect it is actually _less_ work just to ask the server code.
* It's got to be maintained in lock step - if I fix a game mechanics bug in the C++ server which has been faithfully duplicated in a Python client, someone else better be able to change the Python client unless I go and learn Python.
* Keeping the code in one place allows more extensions post Freestars 1.0 - say we want to add an extra LRT (that might interact with other race parameters), calculate fuel usage in a wrap-around universe, or any number of other things that might be more than just reading in a parameter XML rulefile.
* I'm a bit of a purist - the reason I didn't start writing a Java client months ago (when I had more time!) was because I was going to have to duplicate a _lot_ of stuff, and it just felt like a bad idea (because of the reasons above). If anyone else is stuck not writing a client because of the same issues, well here's something I can do that may help encourage them.

Quote:

The Battle Simulator seems to me a very different matter. The actual Battle Engine doesn't need to know about display at all, so long as it can read a fleet configuration and output a logfile (or bit-packed stream or whatever) which a graphic client can render into a battle VCR, or the turn-generating server can use to churn along. So, it could become a closed box, callable from all parties, or even a stand-alone program (which might be easier to do than having it running under any kind of middleware).

Not quite sure what you mean by "closed box, callable from all parties", but one of the things that I want to keep is the ability to do most things offline, which means keeping the battle code local. I agree that there should be a format that the client can render into a battle VCR - why not just use XML rather than a bit-packed stream, etc.?

Report message to a moderator

Re: A modest proposal to make clients easier Fri, 03 February 2006 18:10 Go to previous messageGo to next message
Madman is currently offline Madman

 
Officer Cadet 1st Year

Messages: 228
Registered: November 2003
Location: New Zealand
Kotk wrote on Sat, 04 February 2006 05:18


When i look at current client then i see few more:
* prod que: 1 year prognose (greens and blues) 100 years prognose (blues, blacks and reds)

Green is easy - the client could do that based on knowing available minerals/resources. For the blues, blacks and reds, I'd suggest not worry about it for an alpha version, but it wouldn't be that hard to pass the production queue when asking for planet details, it returns the time estimate of each entry.
Quote:

* fleet: cloaking, gate jump prognose (its too simple) and estimated travel range (its also too simple) also fleet shield and armor would be fine to see.

OK, so better have a way of passing whole designs rather than trying to do it my components. Travel range can have surprising issues: ever had a fleet travel 98.9 light years at warp 7 and take 3 years?
Quote:

* minefield: decay rate
* research: next year prognose
Probably some more ... yes its nice idea to have full list before doing something.

Yes those too, they are pretty easy. The point is that it's a pretty finite list, and isn't needed for every click in the client.

One thing I'm going to try and do whether the XML client idea flies or not is to set up a (C++) API for all the things the client code would need to ask the game engine code so they don't become too tangled.
Quote:

Tools like habitability (with TT CA ally you have far better hab than current client show), opponents scan prognose, growth, battle, packet or bombing simulators ... these things make client lot better.

Yes, once there is a client, I can see lots of stuff to make it nicer, and I'm sure any client API will grow. One thing I'd like to be able to do is put in the best guess at an opponents race and tech levels and ship designs (where you don't know them), and look at things as much as you can from what you think is the opponents point of view - do everything you currently use a separate testbed for.

One of the really nice things with an XML approach is that the client and server data structures don't have to be in lockstep as they would with a C interface - if there is a change to the server where it returns more information, the client can just ignore it. The other way around, if the client uses an old version of the server tht doesn't return all the expected information, it can fail gracefully Smile. The downside is there would have to be a definition document somewhere so the XML messages don't aquire a lot of cruft.
Quote:

My experience show that it is less bad than you say here. For JNI, C++ side has to manage memory for the stuff that it passes. It does not need to be C, just the interface is with C calls. Yes, the battles take lot of room in turn files. For current Stars! it passes too few data so there are battle display flaws and you can even crash the game engine by making too big battles. 100 fleets living 16 rounds each doing 3 phases per round (sounds we talk about megabyte of xml here) so memory management must be flawless on any case no difference if you pass arrays of characters (xml) or arrays of structures.

With C++ having to manage memory for stuff it passes means there has to be a call from the client to the C++ saying "I'm finished with this now". And even if JNI has improved since I last looked at it, it doesn't help if anyone wants to write a client in Python, or decides to write an AI in Scheme or Mercury. By the time you've got marshalling and unmarshalling code between the C and C++ layers, and made sure you've dealt with the memory issues, it's not _that_ much more work to have just done it with XML and have something very language neutral.
Quote:

You say have seen evidence that xml is quick and so i got to trust you. My suspision remains, but it does never hurt when making software. Wink

Fair enough, and I can't show you the project where I've used it because it is closed source (my day job). This is why I'd start with one use of MSQI (the RW, where the feedback is every click) and check it is OK for speed - if it is too slow, I write a post saying so and offering the code if anyone wants to speed it up (I'd want to test it on a 233 MHz Macintosh G3 - I'm not sure we want to aim too much lower than that).
Quote:

I am sure that even C++ UI is not so easy to write and learning to use some crossplatform vidget GUI in the process is smallest of the problems.

Well, it is a bit of an issue in that a learning curve can be a barrier to someone starting. Also, if someone was working on a client 40 hours/week, then learning a new framework isn't _that_ much of an issue. Someone that wants to spend an hour or two on it whenever they can is going to find it a headache though. And often people develop much more willingly and much faster in their language of choice - I can code much faster in Java or Mercury than I can in C++, at least partly because I don't have to think about memory deallocation.

Report message to a moderator

Re: A modest proposal to make clients easier Fri, 03 February 2006 18:58 Go to previous messageGo to next message
gible

 
Commander

Messages: 1343
Registered: November 2002
Location: Wellington, New Zealand

I realise it probably means re writing large chunks of the server , but after reading all of this, it occurs that it would be better to split the server in two - a calculator and a turn processor (and then add a query processor for clients) Depending on how the server is currently organised, this may not be as much work as you might think.

Report message to a moderator

Re: A modest proposal to make clients easier Fri, 03 February 2006 20:59 Go to previous messageGo to next message
m.a@stars is currently offline m.a@stars

 
Commander

Messages: 2765
Registered: October 2004
Location: Third star to the left
Madman wrote on Fri, 03 February 2006 22:27

m.a@stars wrote on Sat, 04 February 2006 04:02

I'd say the better thing would be setting in place *any* kind of graphic Client at all, able to communicate with the current Server to accomplish what simple tasks can currently be accomplished, and then work on the best way to finish linking them both. Best practice here should be less important than say, getting a beta-testing community up and running before Stars itself goes the way of the Dodo. :-/

OK, I agree here - I don't intend my idea to get in the way of someone writing a client in whatever way gets it out the door - what this proposal was _meant_ to do was say "if someone wants to write a client in whatever language they like, I'm prepared to offer a mechanism to get the server to to all the game mechanics".

Even something that just lets you work with the stuff in the turn files would be a good start!


By all means, go ahead. Smile

Madman wrote on Fri, 03 February 2006 22:27

Quote:

But these are not overly costly or even complex calculations, and the worst of them (RW and Hab) are more related than you may believe. So, I think at least part of the Server should indeed be "replicated" or bundled with the Client. But it will be a very simple part, or at least one part which should be pretty fixed, such as the RW and Hab calculations, where only a certain amount of ruleset or game-creation tweaks can be allowed.

There are several issues I see with this:
* There are some subtleties (such as rounding) that do make this code non-trivial. Even the current Stars! occasionally gets it wrong for things like travel distances - do you really think that something travelling 98.97 light years at warp 7 is always going to get there in 2 years? I suspect it is actually _less_ work just to ask the server code.


Don't I know. I spent two weeks nailing down all the little wrinkles of that one, getting the exact fuel usage formula *AND* a nifty trip calculator along the way to overcome exactly that. Sherlock

But the upside of it is, once the math is fully understood, and its roundings explicitly included, it will work just the same whatever the hardware or software or language that runs it. Very Happy

Madman wrote on Fri, 03 February 2006 22:27

* It's got to be maintained in lock step - if I fix a game mechanics bug in the C++ server which has been faithfully duplicated in a Python client, someone else better be able to change the Python client unless I go and learn Python.


Hopefully these formulas will be kept in well-commented places. And, being mostly integer math, they should be far easier to manage than, say, a new command button in the scanner pane.

You of course have a point there.

Madman wrote on Fri, 03 February 2006 22:27

* Keeping the code in one place allows more extensions post Freestars 1.0 - say we want to add an extra LRT (that might interact with other race parameters), calculate fuel usage in a wrap-around universe, or any number of other things that might be more than just reading in a parameter XML rulefile.
* I'm a bit of a purist - the reason I didn't start writing a Java client months ago (when I had more time!) was because I was going to have to duplicate a _lot_ of stuff, and it just felt like a bad idea (because of the reasons above). If anyone else is stuck not writing a client because of the same issues, well here's something I can do that may help encourage them.


How true. Nevertheless, the qwik 'n dirty way of duplicating code, or even ignoring some features (like growth prediction or exact hab values) at first, should get a Client coder past the initial stages easily enough.

Madman wrote on Fri, 03 February 2006 22:27

Quote:

The Battle Simulator seems to me a very different matter. The actual Battle Engine doesn't need to know about display at all, so long as it can read a fleet configuration and output a logfile (or bit-packed stream or whatever) which a graphic client can render into a battle VCR, or the turn-generating server can use to churn along. So, it could become a closed box, callable from all parties, or even a stand-alone program (which might be easier to do than having it running under any kind of middleware).

Not quite sure what you mean by "closed box, callable from all parties", but one of the things that I want to keep is the ability to do most things offline, which means keeping the battle code local. I agree that there should be a format that the client can render into a battle VCR - why not just use XML rather than a bit-packed stream, etc.?


Closed box = a separate process/thread/dll/program, running alongside both Server and Client, with a common interface for both, be it text files, XML, binary files, TCP/IP or telepathy.

As Gible suggested, the "closed box" concept could be applied to the "common" functions, too, so those could also be bundled in the standalone "battle" engine, or into their own standalone "box".

Indeed, I begin to see how your own proposal resembles this, but I'd rather not have to load the whole Server just to answer a dozen of common calls for the Client(s). Smile


[Updated on: Fri, 03 February 2006 21:12]




So many Stars, so few Missiles!

In space no one can hear you scheme! Deal

Report message to a moderator

Re: A modest proposal to make clients easier Fri, 03 February 2006 21:09 Go to previous messageGo to next message
m.a@stars is currently offline m.a@stars

 
Commander

Messages: 2765
Registered: October 2004
Location: Third star to the left
gible wrote on Sat, 04 February 2006 00:58

I realise it probably means re writing large chunks of the server , but after reading all of this, it occurs that it would be better to split the server in two - a calculator and a turn processor (and then add a query processor for clients) Depending on how the server is currently organised, this may not be as much work as you might think.


I hope it doesn't take too much work, as it could help boost the whole project into beta-testing stage. Smile

The "turn processor" and the "calculator" should be fairly independent, anyway. The "query processor" should be an extension of their interface, indeed.



So many Stars, so few Missiles!

In space no one can hear you scheme! Deal

Report message to a moderator

Re: A modest proposal to make clients easier Fri, 03 February 2006 21:24 Go to previous messageGo to next message
Madman is currently offline Madman

 
Officer Cadet 1st Year

Messages: 228
Registered: November 2003
Location: New Zealand
[email

m.a@stars[/email] wrote on Sat, 04 February 2006 14:59]How true. Nevertheless, the qwik 'n dirty way of duplicating code, or even ignoring some features (like growth prediction or exact hab values) at first, should get a Client coder past the initial stages easily enough.

Agree here - anyone wanting to write a client shouldn't wait on me! For any well written client, it could be moved over to my scheme later if that was deemed useful (or when the client coder finds themselves over their head duplicating large amounts of the server, as I suspect would happen).

Madman wrote on Fri, 03 February 2006 22:27

Indeed, I begin to see how your own proposal resembles this, but I'd rather not have to load the whole Server just to answer a dozen of common calls for the Client(s). Smile

What's the particular objection here? The idea would be to load the server once on client start up, and 'invisibly' to the end user (as a library or similar), so as far as they are concerned they are just running the client. In this thread, I've been discussing implementation, not what the player sees.

There's no great cost associated for loading the whole server rather than just the needed bits - very slightly more disk and memory usage for loading the part of the server code not used by the client.

The only other consideration I can think of is security - if the client has its own local version of the server, it would be worth making sure for that instance that any outside communication handled by the server is turned off.


[Updated on: Fri, 03 February 2006 22:33]

Report message to a moderator

Re: A modest proposal to make clients easier Fri, 03 February 2006 22:52 Go to previous messageGo to previous message
Kotk

 
Commander

Messages: 1227
Registered: May 2003
Madman wrote on Sat, 04 February 2006 01:10

Yes those too, they are pretty easy. The point is that it's a pretty finite list, and isn't needed for every click in the client.
Not sure if you get it fully ... the thing does tons of calculations. For example the research prognose you said is "pretty easy" calculates one year build for all queues. click to that research percentage and all queues recalculated to find out new research prognose. Dropped or picked some pop and mining recalc, production recalc and research recalc again. With estimated travel range i meant that (quite useless one) displayed below the battle plan. I am no way supporting "calc it all in your clients, dudes" idea. On the contrary, i believe that needed interface will be quite large and may happen that we are underestimating the task. Very probable that MSQI will need lot more than 5 or 10 different queries. Wink
Quote:

And often people develop much more willingly and much faster in their language of choice - I can code much faster in Java or Mercury than I can in C++, at least partly because I don't have to think about memory deallocation.
Yes, people have different taste. For example me ... i always feel that it takes special treatment in "managed" languages to get some resources deallocated and freed timely. In C++ i feel safe, usually some STL container or smart pointer takes care, no special "dispose()" or other tricks needed. Very Happy Ah whatever, true programmer can write fortran programs in any language and most things including Java, Python or C++ themselves are written in C. Laughing

Report message to a moderator

Previous Topic: Client feature request
Next Topic: Feature questions
Goto Forum:
  


Current Time: Fri May 03 21:20:01 EDT 2024