Home » Stars! 2.6/7 » The Bar » Stars 3
|
Re: Stars 3 |
Sun, 01 June 2014 13:18 |
|
mrvan | | Officer Cadet 1st Year | Messages: 220
Registered: May 2014 | |
|
m.a@stars wrote on Sun, 01 June 2014 13:02XAPBob wrote on Sun, 01 June 2014 16:24I would suggest that the "Logic" code could indeed be a list of the Order Of Events. After all they are independant sections.
Call out for each Event...
Just what my Master Plan said.
I don't disagree per se, I just don't really see technically how that would work. "calling out" within the same language is easy (i.e. have each order of event or group of events a separate modules and call them to progressively construct the 'next turn'). Calling out between languages makes it messy on a number of fronts:
- installation issues (if you make a c program, someone else a .net program and another person a perl script, anyone running the engine would need to compile the C, have a .net and perl interpreter, and get all this working together in a pipeline.
- code re-use. Probably quite a bit of events use similar pieces of code, if only for manipulating the data structure. If we use different languages, this will become really difficult.
What I meant with a "modular architecture" is to split the project up into a number of components, but the "engine" would be one of the components; "client", "automation", "turn processing", "AI" being other components.
If the rest of you are happier to write the whole engine in C, I am fine with that as well, but I think that would be more effort than python. .NET I will not work with due to multiplatform issues, java I would also not prefer.
XAPBob / M.A., what do you guys propose concretely?
Report message to a moderator
|
|
|
Re: Stars 3 |
Sun, 01 June 2014 13:22 |
|
m.a@stars | | Commander | Messages: 2765
Registered: October 2004 Location: Third star to the left | |
|
mrvan wrote on Sun, 01 June 2014 17:20But my feeling is that it will make it rather difficult to keep things consistent and will result in a lot of code redundancy.
Hopefully we can have all (or most) "reusable" functions coded in some module(s) where they can be called by those who need 'em. There might come a time when we'll want to streamline and unify and compact and turbocharge inner workings, but that would be after the initial success, I guess.
Quote:Does anyone really object to writing the engine modules in python so we can combine it into a single 'engine' module?
Well, if you have that much time... Shorter modules like, say, Fuel Usage and Planet Habs, should be easy. Things like the RaceWizard convoluted checks won't be so easy, tho, and I guess there'll be more complex things down the road.
So many Stars, so few Missiles!
In space no one can hear you scheme! Report message to a moderator
|
|
| |
Re: Stars 3 |
Sun, 01 June 2014 13:41 |
|
m.a@stars | | Commander | Messages: 2765
Registered: October 2004 Location: Third star to the left | |
|
mrvan wrote on Sun, 01 June 2014 19:18I don't disagree per se, I just don't really see technically how that would work. "calling out" within the same language is easy (i.e. have each order of event or group of events a separate modules and call them to progressively construct the 'next turn'). Calling out between languages makes it messy on a number of fronts:
Ah ha. We might have been a little carried away when you sold us on the amazing flexibility of Python & friends.
Still, as long as there's "intermediate" files and a way to call external programs sequentially on those, I think there should be a way to make it work.
Quote:- installation issues (if you make a c program, someone else a .net program and another person a perl script, anyone running the engine would need to compile the C, have a .net and perl interpreter, and get all this working together in a pipeline.
Well, we'll provide the source code, but if we also provide binaries for every module, final users should have it relatively easy?
Quote:- code re-use. Probably quite a bit of events use similar pieces of code, if only for manipulating the data structure. If we use different languages, this will become really difficult.
Yup. I'm guessing things like "point-in-circle" and "point-in-poly" will be needed in many different places. So, hmmmm...
About data manipulation, there's always the option of using intermediate data files, in true unix-like tradition. Not pretty but it works. It would be better if each module could be handed down its input data from memory, but that's a bonus that can be solved/refactored later.
Quote:What I meant with a "modular architecture" is to split the project up into a number of components, but the "engine" would be one of the components; "client", "automation", "turn processing", "AI" being other components.
I smell a possible semantics issue here. When I read "modular" I get instantly thinking into splitting the entire Order of Events in very very fine slices, easily coded slices, teensy tiny chunks of functionality that later somehow get linked together by higher-up "engines".
Edit: hmmmmmm, very finely grated cheese in the morning! Victory!
[Updated on: Sun, 01 June 2014 13:44]
So many Stars, so few Missiles!
In space no one can hear you scheme! Report message to a moderator
|
|
| |
Re: Stars 3 |
Sun, 01 June 2014 13:59 |
|
mrvan | | Officer Cadet 1st Year | Messages: 220
Registered: May 2014 | |
|
It is always interesting how disagreements remain hidden until you actually start making things concrete
So... how about this:
1) The "engine" is responsible to turning universe+orders into a new universe.
2) Some other program (e.g. the current starserver stub) is responsible to handling the external interface, validating and merging files, keeping track of state, etc etc.
3) The engine consists of approximately* one module for each item in the order of events
4) Each module takes universe+orders as input and outputs the new universe (optionally plus the orders)
python-pseudocode for the engine (taking http://www.starsfaq.com/order_events.htm as reference)
def process_turn(universe, orders):
universe = scrapping_fleets(universe, orders)
universe = manual_load(universe, orders)
universe = wp0_unload(universe, orders)
....
universe = remote_terraform(universe, orders)
return universe
If you all agree so far, the real questions are:
1) How is state encoded and passed between modules? My vote (at least for the time being) is json, since it is flexible and easy to make changes to. XML is imho too cumbersome to parse and manipulate, plain text is too fragile.
2) How are modules called/coded. I would still really argue against writing modules in different languages* and using system calls, but of course that is possible. However, I would think that the logic in most modules is not actually that difficult, and porting existing e.g. C++ modules to python (or python to C) should be not too much work compared to making sure that existing modules deal with the state as agreed on in (1), which is necessary anyways.
IF we can agree that the modules are to be written in python, I completely agree that every module can be a separate file that is maintained by one person and contains its own coding conventions, style, unit tests, etc.
*)My earlier remark on the seamless mixing of python and C were meant for optimizing a python module that was deemed too slow.
Report message to a moderator
|
|
| |
Re: Stars 3 |
Sun, 01 June 2014 14:40 |
|
XAPBob | | Lt. Commander | Messages: 957
Registered: August 2012 | |
|
json/xml. potatoe, potarto. Both will do the job.
Don't care either way.
Does it seem logical to make a "universe orders" blob out of all input files?
[Updated on: Sun, 01 June 2014 14:41] Report message to a moderator
|
|
|
Re: Stars 3 |
Sun, 01 June 2014 14:53 |
|
m.a@stars | | Commander | Messages: 2765
Registered: October 2004 Location: Third star to the left | |
|
mrvan wrote on Sun, 01 June 2014 19:59It is always interesting how disagreements remain hidden until you actually start making things concrete
I'd rather think about hidden opportunities.
Quote:1) The "engine" is responsible to turning universe+orders into a new universe.
2) Some other program (e.g. the current starserver stub) is responsible to handling the external interface, validating and merging files, keeping track of state, etc etc.
3) The engine consists of approximately* one module for each item in the order of events
Indeed. For sufficiently large values of "one".
Quote:4) Each module takes universe+orders as input and outputs the new universe (optionally plus the orders)
python-pseudocode for the engine (taking http://www.starsfaq.com/order_events.htm as reference)
def process_turn(universe, orders):
universe = scrapping_fleets(universe, orders)
universe = manual_load(universe, orders)
universe = wp0_unload(universe, orders)
....
universe = remote_terraform(universe, orders)
return universe
Essentially
Quote:1) How is state encoded and passed between modules? My vote (at least for the time being) is json, since it is flexible and easy to make changes to. XML is imho too cumbersome to parse and manipulate, plain text is too fragile.
Whatever works best, though I wouldn't discard any of the 3 (Stars! own reports come in CSV format, which is quite manageable). If we can make it so the modules don't even know what kind of format carried the data passed to them, even better.
Quote:2) How are modules called/coded. I would still really argue against writing modules in different languages* and using system calls, but of course that is possible.
That could be enough for starters. Improvements would come later.
Quote: However, I would think that the logic in most modules is not actually that difficult, and porting existing e.g. C++ modules to python (or python to C) should be not too much work compared to making sure that existing modules deal with the state as agreed on in (1), which is necessary anyways.
There's many little things/modules/functions. Lots of logic. Get them working 1st, worry about unification later, me thinks.
Again, I can provide design docs and detailed pseudocode for a bunch of things, but I cannot (yet) write Python, so I'd need to resort to a) handing the work to someone else, or b) code things in C (or Javascript) and let someone else recode them later if needed.
So many Stars, so few Missiles!
In space no one can hear you scheme! Report message to a moderator
|
|
| | |
Re: Stars 3 |
Sun, 01 June 2014 15:43 |
|
mrvan | | Officer Cadet 1st Year | Messages: 220
Registered: May 2014 | |
|
My reason for separating universe and orders is that orders never change during processing, and universe changes all the time. Also, it makes sense conceptually. We could even split into universe, state, and orders later, as planet names, locations and race files are invariant.
Of course, if we stick to json, creating a json file {"universe": <universe>, "orders": [<order1>, <order2>]} is trivial enough.
I've created a new repository, starsengine, that sticks to the design mentioned above
engine.py runs through the whole order of events, calling the separate modules (currently only pop_growth and move_fleets
Each module takes the universe and changes it according to its logic.
- pop_growth iterates over the planets. If the planet is populated, it looks up the owner and multiplies population by growth rate
- move_fleets iterates over the fleet orders, finds the fleet, and moves it.
Both modules have unit test modules in the tests folder that test some assertions on how the engine should behave, e.g. for moving fleets
I'm much happier with this setup*, as the modules and tests are a lot cleaner by sticking to basic json and dumping all the web server / database stuff. Hopefully it also should makes it clearer how to contribute.
PS: don't worry about learning python. It's dead easy for coding simple algorithms. Any programmer that knows e.g. C and JS should be up to speed in no time. I will happily help out as well, but if you look at the examples that I now coded you should be convinced as well
*) I think I was too excited by playing with mongo and flask to think straight about overall design
[Updated on: Sun, 01 June 2014 15:51] Report message to a moderator
|
|
|
Re: Stars 3 |
Sun, 01 June 2014 16:00 |
|
mrvan | | Officer Cadet 1st Year | Messages: 220
Registered: May 2014 | |
|
XAPBob wrote on Sun, 01 June 2014 14:37Keep it fully interpreted. no need for speed here (we're unlikely to have single servers needing to process millions of games (and we could always fire up another server at that point - open source ftw )
Need to read up on json as a format - csv would suffice in my mind - extra outputs are needed though - messages for each player (and maybe battle videos etc, possibly a generic "m file block" return?)
You would need a collection of csv files (for planets, fleets, etc), so that adds some complexity.
JSON is a key-value format, it essentially allows 'objects' to have keys with values that can be primitives, more objects, or lists.
Any csv format, say headers a,b,c and rows 1,2,3 and 4,5,6 can be trivially converted into a (more verbose) json file [{"a": 1, "b": 2, "c":3}, {"a":4, "b":5, "c":6}]. The verbosity is of course a drawback, but it makes it easier to read/write as a human and gives more flexibility for e.g. adding keys later, ommitting certain keys for certain rows, adding embedded 'rows' (for example for ships in a fleet) etc.
Although the format was designed for javascript (json = JavaScript Object Notation), the format also maps quite naturally to python dictionaries and has very good library support in any language I know.
As far as I'm concerned, we can always switch to a more efficient format later if needed, but I suspect it should be OK as there just aren't that many things to keep track of in stars.
For the extra outputs, I think if possible the engine should not mess with video. The result of a battle is a sequence of battleboard token actions (edit: what m.a. refers to as 'battle scripts' above), and visualizing that it the task of a client. Messages can either be part of the universe or kept in a separate file/object.
Edit: On third thought it might make sense to write everything in javascript, since the main clients will probably also be in javascript, facilitating code reuse. But I really, really, hate javascript. So my vote goes for python
[Updated on: Sun, 01 June 2014 16:03] Report message to a moderator
|
|
| |
Re: Stars 3 |
Sun, 01 June 2014 17:40 |
|
XAPBob | | Lt. Commander | Messages: 957
Registered: August 2012 | |
|
mrvan wrote on Sun, 01 June 2014 20:43My reason for separating universe and orders is that orders never change during processing, and universe changes all the time. Also, it makes sense conceptually. We could even split into universe, state, and orders later, as planet names, locations and race files are invariant
Sorry, I meant a glob of all orders, so the processing was done on a glob of orders, not a set of 6,8,... files
Merge the 6 order files, then split according to OOE, then pass each split to the relevant handler.
Report message to a moderator
|
|
|
Re: Stars 3 |
Sun, 01 June 2014 17:49 |
|
mrvan | | Officer Cadet 1st Year | Messages: 220
Registered: May 2014 | |
|
m.a@stars wrote on Sun, 01 June 2014 17:36mrvan wrote on Sun, 01 June 2014 21:43- pop_growth iterates over the planets. If the planet is populated, it looks up the owner and multiplies population by growth rate
Funny that, I'd rather do it the other way around, 1st getting the relevant race params (once) then iterating along that race's planet list. Not only would I avoid uninhabited planets, but it could also be better combined with the rest of the "Production" step.
I would say, have a go at it. I didn't actually really think about optimizing, just about doing something easy but not completely trivial.
(although at the moment there is no 'planet list per player', so the best we can do without changing the universe definition is first getting all races to prevent the repeated lookup)
Report message to a moderator
|
|
|
Re: Stars 3 |
Sun, 01 June 2014 17:51 |
|
mrvan | | Officer Cadet 1st Year | Messages: 220
Registered: May 2014 | |
|
XAPBob wrote on Sun, 01 June 2014 17:40mrvan wrote on Sun, 01 June 2014 20:43My reason for separating universe and orders is that orders never change during processing, and universe changes all the time. Also, it makes sense conceptually. We could even split into universe, state, and orders later, as planet names, locations and race files are invariant
Sorry, I meant a glob of all orders, so the processing was done on a glob of orders, not a set of 6,8,... files
Merge the 6 order files, then split according to OOE, then pass each split to the relevant handler.
Well, at the moment i'm not concerned too much about 'files' yet, and assume that state is passed around as python "json" dictionaries. This will become more relevant if we decide to include external modules, but it should be quite easy to change at any point.
Report message to a moderator
|
|
|
Re: Stars 3 |
Sun, 01 June 2014 18:07 |
|
m.a@stars | | Commander | Messages: 2765
Registered: October 2004 Location: Third star to the left | |
|
mrvan wrote on Sun, 01 June 2014 22:00You would need a collection of csv files (for planets, fleets, etc), so that adds some complexity.
Not too much, I hope. There's a bunch different "data tables" that need to be considered, and it makes sense to keep 'em apart, even if it's just as independent chunks of the same json file.
* race params (all races together? makes sense, but sounds... weird)
* Univ params, including the techtree (plus hull defs & fuel tables). PRT and LRT params. HWs?
* fleets (incl Ship designs), planets, diplo settings, research status, battle plans, default Qs
* messages? (at least player-to-player) Multi-turn orders? Minefields. Packets. Transport Orders.
* battle reports, MTs, WormHoles, comets, player_orders, salvage, scores, etc.
Quote:JSON is a key-value format, it essentially allows 'objects' to have keys with values that can be primitives, more objects, or lists.
Any csv format, say headers a,b,c and rows 1,2,3 and 4,5,6 can be trivially converted into a (more verbose) json file [{"a": 1, "b": 2, "c":3}, {"a":4, "b":5, "c":6}]. The verbosity is of course a drawback, but it makes it easier to read/write as a human and gives more flexibility for e.g. adding keys later, ommitting certain keys for certain rows, adding embedded 'rows' (for example for ships in a fleet) etc.
That kind of flexibility will be very useful!
So many Stars, so few Missiles!
In space no one can hear you scheme! Report message to a moderator
|
|
| | |
Re: Stars 3 |
Sun, 01 June 2014 18:26 |
|
mrvan | | Officer Cadet 1st Year | Messages: 220
Registered: May 2014 | |
|
m.a@stars wrote on Sun, 01 June 2014 18:07mrvan wrote on Sun, 01 June 2014 22:00You would need a collection of csv files (for planets, fleets, etc), so that adds some complexity.
Not too much, I hope. There's a bunch different "data tables" that need to be considered, and it makes sense to keep 'em apart, even if it's just as independent chunks of the same json file.
* race params (all races together? makes sense, but sounds... weird)
* Univ params, including the techtree (plus hull defs & fuel tables). PRT and LRT params. HWs?
* fleets (incl Ship designs), planets, diplo settings, research status, battle plans, default Qs
* messages? (at least player-to-player) Multi-turn orders? Minefields. Packets. Transport Orders.
* battle reports, MTs, WormHoles, comets, player_orders, salvage, scores, etc.
Yes, there will be quite a lot of stuff to store. Maybe race files should go separate, and I guess the tech tree is certainly separate (since it is not really part of the 'state'), but for the moment I would think it's best to throw everything together and refactor as needed down the line.
It will require a bit of puzzling to think of how best to deal with things like fleets and ship designs. I guess things like deleting and creating designs are all orders, as are things like merging a fleet and giving the new fleet orders. However, we need to think of how to give orders to a newly created fleet or build ships of a newly created design. Does the client assign an ID? How do you prevent clashes? UUIDs? player-specific fleet/design numbers? And I suppose that something like 'deleting a design' is even earlier in the order of events than scrapping. But we will run into these and related issues soon enough
Report message to a moderator
|
|
| | | |
Goto Forum:
Current Time: Sun May 19 10:35:20 EDT 2024
|