Home World Forum
Stars! AutoHost web forums

Jump to Stars! AutoHost


 
 
Home » Stars! Clones, Extensions, Modding » Stars! Nova - Development » Orders
Orders Wed, 06 July 2011 23:03 Go to next message
Aeglos is currently offline Aeglos

 
Chief Warrant Officer 1
Stars! Nova developer
Stars! Nova developer

Messages: 142
Registered: May 2011
Location: Chile
The great Id refactoring is practically done, and the Intel file is well underway to being properly structured.

So, let's talk Orders now; Let's clear some ideas for it's future restructuring, if any.

Right now, the Orders file is nothing more than an updated State; it sends the state read from the intel plus any changes the client made. I was under the impresion that the idea was to change this to send some kind of order commands that the server would process, instead of merging each client's state to it's master copies and then process fleets, stars, etc.

Right now the server keeps a record of each client's valid state for the start of each turn. Due to this, It's easier for the server to check for errors and cheats than before, but it's not flawless. Checking fleet positions for example; it can be done but it would be awkward to loop all fleets and compare their coords for each player when reading their turns. We would need to do this for stars also, and for almost all of their fields, else a client can silently modify some data from objects and it would be merged into the server's master copies.

Ideally the client would send only commands like "add waypoint to fleet #xx", "remove item from star #xx's queue" or "activate mass driver on star #xx". There should be a dinstinct and finite number of commands a client can send... in fact, the amount of commands for Stars seems to be very small. I can think of this command "types":

  • Add/remove waypoint
  • Merge/split manually
  • Modify cargo manually
  • Add/remove item from production queue
  • Mark/Unmark a star for leftover resources only
  • Activate Mass Driver (could be waypointed since starbases are fleets)
  • Change research target
  • Change research budget
  • Add/edit/remove a design (edit could be a remove-add combo).
  • Add/edit/remove a battleplan
  • Change a relation


And that seems to be it... I can't think of anything else. So the list is pretty small to be passing the whole state down to the server.

This could easily be achieved with a bunch of concrete order implementations that inherit a base Order class.

Thoughts?



[Updated on: Wed, 06 July 2011 23:17]

Report message to a moderator

Re: Orders Thu, 07 July 2011 04:57 Go to previous messageGo to next message
Daniel is currently offline Daniel

 
Chief Warrant Officer 3
Stars! Nova developer
Stars! Nova developer

Messages: 179
Registered: April 2006
Location: Nowra, Australia
I agree with the idea of orders being distinct from state.

I think the following additional orders need to be considered:

  • Fleet ## Start/Stop remote mining.
  • Fleet ## lay mines for-X-years, forever, stop-laying.
  • Send message to player(s)/empire(s) (and/or self).
  • Set waypoint zip-order
  • Transfer fleet ## to empire/player/race #.


There may be more, but that is all I can think of for now.



Have fun.

Report message to a moderator

Re: Orders Thu, 07 July 2011 05:18 Go to previous messageGo to next message
Aeglos is currently offline Aeglos

 
Chief Warrant Officer 1
Stars! Nova developer
Stars! Nova developer

Messages: 142
Registered: May 2011
Location: Chile
Aren't remote mining, fleet transfers and mine laying waypoints too? At least in Stars! those commands are made on the waypoint selection list.

ZipOrdrs could be easily treated as Waypoint "Packs", so not much different from them I think.

Report message to a moderator

Re: Orders Thu, 07 July 2011 05:39 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
I agree that not all the state needs to be sent - however keeping track of orders like this will get tricky. Orders can maybe consist of state that the user can change.

e.g. for Fleets the position etc. can't change. However the Name and Waypoint data can change. The client can just write the fleet name and new waypoint data into the orders which the server can check and apply. This is a lot eaiser than trying to work out what waypoints have changed and then applying that diff at the other end.

Merge/Split is tricker. Maybe the fleets sent above also contain current ships and cargo levels. The server can see what fleets it sent the client and can find what fleets exist now, and as long as total cargo & ships at any given position is consistent then it's all good

In fact for fleets maybe it is just simpler to dump all the Fleet State into the orders, then the server just looks at each one and applies relevant changes and ignores things like position. Or if it sees that a fleet has magically moved then it can flag that player as cheating?

Again for stars send the whole production queue. The server can check that the planet can produce that stuff (i.e. make sure that Starbase exists before trying to build a ship)

Report message to a moderator

Re: Orders Thu, 07 July 2011 06:31 Go to previous messageGo to next message
Aeglos is currently offline Aeglos

 
Chief Warrant Officer 1
Stars! Nova developer
Stars! Nova developer

Messages: 142
Registered: May 2011
Location: Chile
The problem with that approach is that the client shouldn't modify it's state at all; it should just tell the server to modify it instead, otherwise it would force a lot of checking server-side for valid values since those changes are not under the server's control.

For the split/merge it would need to check not only the amount of ships and cargo, but also the cargo composition, else the client could magically change colonists into ironium. It would have to check each ship and it's components lest they changed, their individual damage amounts, etc, and see if it all matched the previous fleet values.
The other option of the server just cherry picking which variables to update fails for cases like split/merge where almost all of the values need to be updated.

For the production queues something similar happens, it would have to check each production item's data individually. Consider this; a client which has an item at 50% production edits his orders so that it's at 70%. The server has to check each item to see if the progress has changed, and only allow for decreases in progress, in case someone for example, removed that 50% item by mistake and then added it again, reseting it to 0%; it's a valid order that the server has to accept, so it would need to check all items for special cases like that where some changes are allowed and others not.

It just feels easier to send to the server "merge fleet Y & Z" or "add item to X stars queue", where the checks would be far less. Merging/Splitting would need only to check for same owners and same position (not even nessesary for split) since the server is sure the fleet data has not been tampered with. Production queues would only need to check for component/design availability and star capability. Waypoint modifications would only need to check owners, and if deleting, that the waypoint exists already. (Since editing is simply a remove-add combo).



Report message to a moderator

Re: Orders Thu, 07 July 2011 06:54 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
I think you've missed what I'm saying. The server doesn't just take the state and overwrite it, it only takes and verifies what the client is allowed to changed.

With the split/merge I absolutely meant it checks the total ammount of individual cargos. However it just get's the ship IDs that are now in each fleet - it never needs to look at the components as they cannot change. It just modifies it's state based on the values sent

For production queues again it only wants to look at the queue now. In the example you mention I'd probably want the server to realise and produce the item - however each production item added can be given an Id of course.

Aeglos wrote on Thu, 07 July 2011 06:31


It just feels easier to send to the server "merge fleet Y & Z" or "add item to X stars queue", where the checks would be far less. Merging/Splitting would need only to check for same owners and same position (not even nessesary for split) since the server is sure the fleet data has not been tampered with. Production queues would only need to check for component/design availability and star capability. Waypoint modifications would only need to check owners, and if deleting, that the waypoint exists already. (Since editing is simply a remove-add combo).


I agree it sounds better this way, and propbably would be better. However generating and tracking all those orders seems hard. Plus currently you can't save a turn and then re-open it to see what you've done, so all the changes the orders make need to be shown somehow. This just feels like a lot of work no?

Report message to a moderator

Re: Orders Thu, 07 July 2011 07:02 Go to previous messageGo to next message
Aeglos is currently offline Aeglos

 
Chief Warrant Officer 1
Stars! Nova developer
Stars! Nova developer

Messages: 142
Registered: May 2011
Location: Chile
Indeed I'm missing the point. I don't follow the split/merge for example.

What I understood is that for merging, for example, the client merges the fleets, and sends it's state with the fleets already merged. So the server checks if it's valid and updates it's own copies. But to do so, it needs to first find the two fleets that were merged, and double check that everything is still the same; designs, components, cargo amounts, damage amounts.

Report message to a moderator

Re: Orders Thu, 07 July 2011 07:49 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
Aeglos wrote on Thu, 07 July 2011 07:02


What I understood is that for merging, for example, the client merges the fleets, and sends it's state with the fleets already merged. So the server checks if it's valid and updates it's own copies. But to do so, it needs to first find the two fleets that were merged, and double check that everything is still the same; designs, components, cargo amounts, damage amounts.



Well to find the fleets it can use ship IDs as a ship can only ever be in one fleet. Once it's done that it just has to check what cargo is in the new fleets and make sure that the totals are the same as previously, and that each fleet can carry what it now has. Damage ammouns, designs and components CANNOT be changed by the client so it doesn't check those, it just ignores them, throws them away in fact, and keeps what it already has from what it sent to the client.

I.e. it takes the ships from the previous turns, puts them into new the fleet objects, adjusts the cargo and fuel etc.

[edit to remove last edits]



[Updated on: Thu, 07 July 2011 07:59]

Report message to a moderator

Re: Orders Thu, 07 July 2011 07:55 Go to previous messageGo to next message
Aeglos is currently offline Aeglos

 
Chief Warrant Officer 1
Stars! Nova developer
Stars! Nova developer

Messages: 142
Registered: May 2011
Location: Chile
But there is a flaw there...

If the client sends the fleet already merged and the server uses that as it's data source, then it needs to validate every single field, as the client could have modified those.

If it reads the merged fleet and then updates it or re-creates it fully or partially from it's master copies, then there's no point in the client merging it in the first place since the server has to do it again anyway after validating, hence the "merge order" makes more sense.

Report message to a moderator

Re: Orders Thu, 07 July 2011 08:02 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
Aeglos wrote on Thu, 07 July 2011 07:55

But there is a flaw there...

If the client sends the fleet already merged and the server uses that as it's data source, then it needs to validate every single field, as the client could have modified those.



My point is that the server NEVER READS those fields. Yes - it essentially creates 2 new fleets and moves the ships from it's own data store across. So yes the server is re-doing the client merge. Yes, this harder than just processing a merge order, however it actually may be easier than producing that merge order in the first place.

Just to re-interate, I'm mainly suggesting this as the easiest way from where [i think] we are.

Leave that to one side for now and let's explore the other option how are we proposing to track orders as the client makes them, and how does the UI show the last known state + orders?



[Updated on: Thu, 07 July 2011 08:03]

Report message to a moderator

Re: Orders Thu, 07 July 2011 08:17 Go to previous messageGo to next message
Aeglos is currently offline Aeglos

 
Chief Warrant Officer 1
Stars! Nova developer
Stars! Nova developer

Messages: 142
Registered: May 2011
Location: Chile
Alrighty then.

I was thinking of adding a collection to ClientState which holds Order objects. We simply add/remove them there when needed.

Order would be a common interface or base class, and there would be concrete implementations; WaypointOrder, ProductionOrder, MergeOrder, RenameOrder, etc. Or separated by a Type variable.

Showing state+orders is the trickier part. I can think of two things right now.
Parse the orders after the state, which would be similar to what the server should do. They could use a common order parser, or do it on a per case basis on the client.
Or, hold two state copies on the client, the initial and an updated one, and for each change we both update the state and issue and order. The client sends only orders down to the server, and discards it's changed state each turn in favor for the one on the intel. This feels terribly redundant, but could be easier than the first approach.

The key thing that i'm holding on to is the principle that we shouldn't trust anything that comes form the client; we should expect it to lie, cheat and malverse it's state and orders every time, that's why I'm mostly against sending state to the server. But, as always, I'll go with the majority vote, if for whatever reason implementing orders turns out to be unfeasible.

Report message to a moderator

Re: Orders Thu, 07 July 2011 09:32 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
Currently the whole UI is driven from the client side state and changing that would be a PITA. Having 2 states is an option, and then we'd have to modify the code everywhere to add orders.

Of course there may be orders that change previous orders, e.g. you re-split a merged fleet, or you delete all a fleets way points and re-add a load more. And then do the same again. No problems really of course as you can just grow the list of orders indefinately as each one must make sense if you play them back in the same order.

Another option would be produce orders from the old and new states. I.e. the client would just udpate a new state and then on submit it would work out the orders needed to transform one to the other. However in this case the server would need to verify it all anyway so the client may as well send all it's state over and have the server work out the orders.


In fact that's a better way to think of what I'm suggesting. The Server doesn't just take the state the client sends as the The Truth. It looks at the OLD state it sent and the ORDERS state it gets back and essentially generates what orders were made, checks they are valid, and applies them to the OLD state to produce a NEW state for the next turn. (It probably never actually produces a List<Order> as it were, but that's logically what it's doing). Does that now make more sense of what I was suggesting?

Aeglos wrote on Thu, 07 July 2011 08:17


The key thing that i'm holding on to is the principle that we shouldn't trust anything that comes form the client; we should expect it to lie, cheat and malverse it's state and orders every time


Absolutely agree - we have to assume the client is going to cheat as someone will, which is why I think of the server working out the orders on the fly and checking them,

Aeglos


But, as always, I'll go with the majority vote, if for whatever reason implementing orders turns out to be unfeasible.



Likewise I'm happy to be persuaded either way, or go along with any decision made. There is some level of pragmatism in what I'm suggesting given where we are in the code base. If we were going from scratch then it may be a different discussion Smile




Report message to a moderator

Re: Orders Thu, 07 July 2011 10:14 Go to previous messageGo to next message
Daniel is currently offline Daniel

 
Chief Warrant Officer 3
Stars! Nova developer
Stars! Nova developer

Messages: 179
Registered: April 2006
Location: Nowra, Australia
I think that either approach is valid and each has advantages and disadvantages. Storing and tracking the steps taken by the player to manage fleets and production queues would eliminate the posibility of accidentilly copying state that is invalid, but may be very complex to implement and to debug.

Taking an "orders state" and using it to determine what was changed and if that was vaild must be done very carefully to avoid copying invalid state, however it may be easier to achieve from the current code base and simpler to debug/maintain.

Who is going to do this? I suggest that whoever wants to take the lead on implementing one of the "verified orders" options pick the one they want to work on. We do not need the best solution. We need a workable solution that someone has the time and interest to implement. It may be that a more complex solution is more interesting and therefore what you want to spend your time doing (ala XML). Alternatively you may have a clear idea how to do it your way and that is the way you are going to do it.

This is not a democracy, it is a meritocracy. Discussion of options and ideas is encouraged, but ultimately it is up to you as a developer to decide what you will implement. If your solution does not work well and someone else has time and skill to do it better, they will. If your solution does work well then we have plenty of other things to work on.



Have fun.

Report message to a moderator

Re: Orders Thu, 07 July 2011 10:51 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
Daniel wrote on Thu, 07 July 2011 10:14


Who is going to do this?


Well I guess me or Aeglas Wink I'm happy to work on either solution - I think we're just discussing which we want to attempt. And yes I would much rather do this over a few beers with a a load of paper to draw on but the forums will have to do sadly.


[Updated on: Thu, 07 July 2011 10:52]

Report message to a moderator

Re: Orders Thu, 07 July 2011 14:39 Go to previous messageGo to next message
Aeglos is currently offline Aeglos

 
Chief Warrant Officer 1
Stars! Nova developer
Stars! Nova developer

Messages: 142
Registered: May 2011
Location: Chile
We could have virtual beers Cheers

In any case I think this change is not something we should start right now, or even soon for that matter. There are plenty of things to do before this, and I was thinking of aiming for a 0.6 or 0.7 release with it. So let's take time to evaluate the options, and to devise new ones if we can. I'd like to hear what evil has to suggest, if anything, as he also has good insights on how Nova works.

When the time comes though, I could gladly devise some kind controversial order parser and see if it works.

This is also heavily related to the pending server restructuring too.

Report message to a moderator

Re: Orders Thu, 07 July 2011 19:53 Go to previous messageGo to next message
evild00d is currently offline evild00d

 
Crewman 2nd Class
Stars! Nova developer
Stars! Nova developer

Messages: 14
Registered: July 2009
Location: Norway
This is an important architectural decision so getting it "wrong" just means that changing it later will cause a lot of pain. I don't like pain Crying or Very Sad I just want stuff to work Smile (like it's that easy)

That said, I'm very much in favor of the proposed command/order solution, but I'm not saying that this is necessarily THE RIGHT WAY(tm).

In this particular case (client-server communication, validation, etc.) storing and passing around state and trying to validate it will likely make things difficult to maintain in the long run. The client- and server-side at some point need to know the state, but conveying a change as a command allows us to capture the player's intent.

With commands (orders if you wish), one of the benefits on the client side would be that undo/redo support would be easy to implement (just one/two stacks of commands and code to reverse commands). On the server side it would make testing, debugging and validation easier. The entire sequence of commands (or resulting events) made during a game could be stored and used as a basis for auditing, running it through external analysis tools, replays, retroactively checking for potential cheats used in a game if a cheat/hack/exploit is published some time after the game completed.

Implementing commands and some kind of "event sourcing" (http://martinfowler.com/eaaDev/EventSourcing.html if you will) is a story for another time Rolling Eyes

Report message to a moderator

Re: Orders Fri, 08 July 2011 05:51 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
I agree if we were designing from scratch we might do it that way. But I think it's a lot work from where we are for a team with not that much time (look how long it's taken to get to this point)

Also one thing with generating orders which tries to keep state in 2 places the same usually runs into problems when something get's missed and the states get out of sync. This is absolute <explitive deleted> nightmare to debug, and I speak from bitter experience. Having a state to say "make it look like this" is then much easier to compare and see where the issues are.

So I'm saying the opposite about maintainablity Wink

That said... having an undo list is attractive. In this case every action that can change the state would do so by creating an "action" on the orders list - such that taking the original state and applying that action leaves you in the new state. Also an action has to know how to UNDO itself, by either crudely caching the entire previous state, or at least know what to change back (e.g. for a waypoint change record the previous waypoints).

This way the client state is always LATEST and can be displayed as it is now, and orders can be undone, redone and sent to the server. There's the added bonus that you can display a list of what a player has done as well this turn.




[Updated on: Fri, 08 July 2011 05:51]

Report message to a moderator

Re: Orders Tue, 19 July 2011 05:57 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
Interesting info in this thread here

i.e. this does just keep a log of player actions. Maybe we should try and move to that?

At a guess the easier way will be to get a state from the server which the client modifies as well as building up an actions log. Then on the next turn the client receives a whole new state from the server and uses that.

The only issue then is with historical information - i.e. does the Server only send fleets and stars the player can see This Turn and then the client keeps them, or shoudl the server keep what it's sent so far and resend each time? (Actually this may only matter for Star Reports, as for fleets we may only need the current known reports?)

Report message to a moderator

Re: Orders Wed, 20 July 2011 17:34 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
Just had a thought on this idea which makes me happier with it... and makes it easier to make it an incremental change.

The Orders would presumably have base class to derive from, then you'd have (e.g.) WaypointOrder, or maybe more specific like WaypointAddOrder etc. inherit from that.

The orders base class could then have an ApplyToClientState method - passing in an existing state and the order modifying it for display.

This means that we can go through the user actions making orders and moving any client state changes into the specific order Apply method.

Orders can be then saved to disc, and on re-start if it's the same turn then the old state loaded and the orders re-played in memory to bring it up to date. This also adds an inherent undo mechanism.

Don't know why I didn't all see this before like I think you were trying to tell me.


Orders on the server are done in a certain order aren't they? So this may not work in the same way server side - but the client side changes can be made, and "submit" then can still save the new state to the orders file as it is now alongside the new style orders - and then we can start on the server side changes!

Report message to a moderator

Re: Orders Wed, 20 July 2011 18:43 Go to previous messageGo to next message
Aeglos is currently offline Aeglos

 
Chief Warrant Officer 1
Stars! Nova developer
Stars! Nova developer

Messages: 142
Registered: May 2011
Location: Chile
Indeed that was what I (at least) was trying to convey (ApplyToClientState would be the concrete implementation of the "parser" notion I mentioned for example). Very Happy

Reminds me I have to finish the fleet reports still.

I checked the post linked above, and it seems that the separate report/full object system we chose in the end is the same as Stars (FleetBlock/PartialFleetBlock and PlanetBlock/PartialPlanetBlock). At least we are going in the same direction after all, which is good.

On an interesting note, there's confirmation that the manual cargo orders are treated differently from waypoint zero ones (Blocks 1 & 2 vs Waypoint blocks). That will indeed pose a challenge, as we need to prevent players from changing w0 orders to manual ones by editing the .order file, if those indeed trump w0 ones on the order of events.

Report message to a moderator

Re: Orders Fri, 19 August 2011 16:06 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
So after a break.... I thought I might try and start on the new Orders system? If that's OK with everyone?

Report message to a moderator

Re: Orders Fri, 19 August 2011 17:54 Go to previous messageGo to next message
Daniel is currently offline Daniel

 
Chief Warrant Officer 3
Stars! Nova developer
Stars! Nova developer

Messages: 179
Registered: April 2006
Location: Nowra, Australia
I have not done much testing since the recent round of reorganising code. I would suggest you try running the game through before jumping into orders to make sure the current baseline is stable enough.



Have fun.

Report message to a moderator

Re: Orders Sat, 20 August 2011 02:53 Go to previous messageGo to next message
Aeglos is currently offline Aeglos

 
Chief Warrant Officer 1
Stars! Nova developer
Stars! Nova developer

Messages: 142
Registered: May 2011
Location: Chile
There are still some bugs/crashes to tackle, mainly with the battle engine & the report system and leftover reports which have no actual fleet counterpart.

I've been thinking how to best approach it; wether to remove reports during the battle step alongside fleets, or in the final "scan step" of the turn where all scanning takes place.

One issue i've pondered is if the scanning turn step is actually the right choice. It works, but perhaps fleets should scan and generate reports on the fleet update step instead. Same goes for Stars.


[Updated on: Sat, 20 August 2011 02:55]

Report message to a moderator

Re: Orders Sat, 20 August 2011 04:34 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
Daniel wrote on Fri, 19 August 2011 17:54

I have not done much testing since the recent round of reorganising code. I would suggest you try running the game through before jumping into orders to make sure the current baseline is stable enough.



I was just coming on the forums to say exactly that as well - time to get testing Smile

Report message to a moderator

Re: Orders Wed, 22 February 2012 03:12 Go to previous messageGo to previous message
Daniel is currently offline Daniel

 
Chief Warrant Officer 3
Stars! Nova developer
Stars! Nova developer

Messages: 179
Registered: April 2006
Location: Nowra, Australia
It looks like work has been started on a new orders system, however this has temporarily resulted in almost no orders being passed to the server (the .orders file is almost empty). I think this is because the implementation of the new commands is incomplete at this stage. I also suspect that this is not going to be finished in a very short time.

I think that while this is outstanding it will prevent any other work progressing. This is a problem as not everyone can work on this part of the game at the same time.

I suggest a work around so that turn generation continues to work properly whilst the new commands are being implemented. This would be to have the client produce two .orders files, one using the old state dump and one using the new command system. Then use a flag in the server to decide which one to load. This flag can be set to the new orders for development of the command system and left with the old orders for other work.

Am I understanding what is going on here properly? Is there any issue with doing this? I am going to set this up in my local copy and will commit it if there are no objections.



Have fun.

Report message to a moderator

Previous Topic: New Waypoint System
Next Topic: More Testing
Goto Forum:
  


Current Time: Thu Mar 28 17:24:11 EDT 2024