Home World Forum
Stars! AutoHost web forums

Jump to Stars! AutoHost


 
 
Home » Stars! Clones, Extensions, Modding » Stars! Nova - Development » Need to move away from using names as IDs
icon4.gif  Need to move away from using names as IDs Fri, 24 June 2011 20:40 Go to next message
FurFuznel is currently offline FurFuznel

 
Lt. Junior Grade
Stars! Nova developer
Stars! Nova developer

Messages: 437
Registered: November 2002
Location: New Brunswick, Canada
Good day to all of you!

While working on the ProductionQueue dialogue I ran into bug 3030319 again. This bug is caused by the use of race "Names" and design "Names" as ID codes. As we are going through some major overhauls of the code right now I wanted to bring this up in case anyone has an easy solution to the problem.

I had thought about the possibility of assigning player numbers when the game is first generated and then using those in place of the race "Name" as that would be a unique value. I am not sure if this could be exploitable by players though if they hack their turn orders files.

For designs, it might make more sense to have a simple counter that keeps track of the number of designs created by that player (or Empire as we refer to them now - which I really like). Then the design number could be used to refer to the designs (designIDs). I am not sure how the server could check for valid designIDs though as they would be generated by the player and submitted as part of their orders.

Just some things for people to consider as they go about making huge changes to the code.

I checked in Stars! and I can have two Starbase designs that are both named: "Amazing Starbase". Both designs show up in the queue, and if I build one the other is still available in the queue. Right now in Stars! Nova if I name two Starbase designs with the same name only the second one exists in the Design Manager and only one shows up in the Available Designs list in the production queue.



Shadallark <==> FurFuznel
Mental anguish is for those who choose to think - FurFuznel
running Mac OS X 10.6.7

Report message to a moderator

Re: Need to move away from using names as IDs Fri, 24 June 2011 21:28 Go to previous messageGo to next message
Aeglos is currently offline Aeglos

 
Chief Warrant Officer 1
Stars! Nova developer
Stars! Nova developer

Messages: 142
Registered: May 2011
Location: Chile
Yes, this also extends to races. Sorting everything manually by "RaceName/some#" is cumbersome and hard to mantain. Preferably this variable should be as automatic as possible.

The Item base class has a Key property which returns an already formed <Owner/Name> string for use on collections but it's used mostly on the Design Manager for now and that still poses problems. But it's a starting point. Players are also already asigned a number at game creation.

We can either use something like racename+playernumber to identify each empire, or generate a unique empire number and use racename+empirenumber, which I'm leaning towards to in case we want to implement multiple players per empire as Evil suggested. That solves the duplicate race issue I believe.

EmpireData should hold ShipDesignID (0 to 16), StarbaseDesignID (o to 10) and FleetID (0 to 512) counters to use as part of the unique keys for designs and fleets, and to also keep track of the amount of them.

I'd rather not do this myself yet though, as the refactoring I'm doing now is big enough. If someone else does it I'll still merge it ot the branch and update any conflicts that may arise.

EDIT: This also extends to the methods of comparison we are using now. We use Race.Name to check for ownership everywhere, we should change it to the new ID method we choose to apply, as Owner should be a Key ID and not a simple race name.


[Updated on: Fri, 24 June 2011 22:34]

Report message to a moderator

Re: Need to move away from using names as IDs Sat, 25 June 2011 06:32 Go to previous messageGo to next message
FurFuznel is currently offline FurFuznel

 
Lt. Junior Grade
Stars! Nova developer
Stars! Nova developer

Messages: 437
Registered: November 2002
Location: New Brunswick, Canada
Aeglos wrote on Fri, 24 June 2011 22:28

EmpireData should hold ShipDesignID (0 to 16), StarbaseDesignID (o to 10) and FleetID (0 to 512) counters to use as part of the unique keys for designs and fleets, and to also keep track of the amount of them.


This should be controllable by a setting selectable at game creation. Yes we are creating a clone of Stars! but we are also intending on improving and expanding on it later.

At game creation there should be different options that can be selected about how close or how different from Stars! that particular game is going to be. Therefore, to be ready for that future the maximum number of ship designs, starbase designs, and fleets should be variables rather than hard coded.

Shadallark



Shadallark <==> FurFuznel
Mental anguish is for those who choose to think - FurFuznel
running Mac OS X 10.6.7

Report message to a moderator

Re: Need to move away from using names as IDs Sat, 25 June 2011 07:09 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
We so need to move away from names as keys. However if we're using Empire for players then we no longer need the race as part of the key - that just becomes metadata about the empire. The empire ID is ennough to uniquely identify it.

We should make new composite keys for fleets, designs etc. This should be some form of Empire ID and FleetID, DesignID etc. as the client needs to be able to allocate them as well (e.g. when spliting fleets it needs to make a new FleetID so the current idea of the Server keeping a global ID counter is flawed)

We can either make strings (like "1/1", "1/2") I tend to prefer ints - and use the (e.g) high 8 bits for empire ID and the rest for the counter. This allows for 128 empires with 16 million IDs each. To get the Empire ID just && with 0xFF000000 and the ID && with 0x00FFFFFF. I.e. start numbering Empires at 0x01000000 and go up from there (no need it has to start at 1!)

Report message to a moderator

Re: Need to move away from using names as IDs Sat, 25 June 2011 07:10 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
I would also suggest Aeglos does what he's doing first and then we start to move to whatever system we end up deciding here?

Report message to a moderator

Re: Need to move away from using names as IDs Sat, 25 June 2011 07:17 Go to previous messageGo to next message
Aeglos is currently offline Aeglos

 
Chief Warrant Officer 1
Stars! Nova developer
Stars! Nova developer

Messages: 142
Registered: May 2011
Location: Chile
Would we actually gain anything by using ints like that? While I love bit operations (specially on C/C++), I'm not sure It would benefit us over a plain string of ints. I don't see a performance issue there, and Xml files would be much harder to inspect manually and look at ID relationships.

I do loathe the "/" concatenator though. Perhaps a dash or a dot?

EDIT: Yes, better the ClientState is cleaned first; less collections to update. I was tempted to start doing it today, but then the branch would get insane so I refrained myself.


[Updated on: Sat, 25 June 2011 07:18]

Report message to a moderator

Re: Need to move away from using names as IDs Sat, 25 June 2011 07:28 Go to previous messageGo to next message
Daniel is currently offline Daniel

 
Chief Warrant Officer 3
Stars! Nova developer
Stars! Nova developer

Messages: 179
Registered: April 2006
Location: Nowra, Australia
Musmuris wrote on Sat, 25 June 2011 21:09


We can either make strings (like "1/1", "1/2") I tend to prefer ints - and use the (e.g) high 8 bits for empire ID and the rest for the counter. This allows for 128 empires with 16 million IDs each. To get the Empire ID just && with 0xFF000000 and the ID && with 0x00FFFFFF. I.e. start numbering Empires at 0x01000000 and go up from there (no need it has to start at 1!)




As we do not need to squeeze out every last bit of memory here we can avoid the trickery of bit stuffing and use two ints, one for the empireId and one for the fleetNumer/shipDesignNumber/starbaseDesignNumber. Otherwise we might run into trouble when people accidently use logical-and instead of bitwise-and.



Have fun.

Report message to a moderator

Re: Need to move away from using names as IDs Sat, 25 June 2011 08:00 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
Not the memory at all. Or speed as on a modern computer this is largely irrelevant for a game like Nova. But when strings are used in dictionaries it has to build the hashcode for a string on lookup. Computers just handle 32 bit ints better

It's more just that int ID = EmpireId || fleetID just "feels" nicer than String ID = EmpireID.ToString() + "." + FleetID.ToString() or String.Format("{0}.{1}", EmpireID, FleetID);

It looks even uglier trying to pull them apart with .Split and Int32.TryParse. But we can make those a Utility method somewhere.

In XML is "1/1" much easier to read than 0x01000001 (or even 0x0A0012FE which is just empire 0x0A and Fleet 0x12FE? rather than 10/4862). Once you're away from names like
"Silicanoid/10" you've lost the ease of reading anyway haven't you?

I would just naturally fall more to an Int key or ID rather than a string one unless there's a good reason. But I'm not going to argue this one too hard here Smile

Crucially we need to split internal ID from Display name as i've said before. This will need many many UI changes though where the name is used now in Combo boxes etc. rather than an object to wrap the id/name


[Updated on: Sat, 25 June 2011 08:02]

Report message to a moderator

Re: Need to move away from using names as IDs Sat, 25 June 2011 18:35 Go to previous messageGo to next message
Daniel is currently offline Daniel

 
Chief Warrant Officer 3
Stars! Nova developer
Stars! Nova developer

Messages: 179
Registered: April 2006
Location: Nowra, Australia
Ahh, I see. You want a single key. Yes I agree that a numeric key makes sense over a string. I suggest we implement Key, EmpireId and FleetId/DesignId/etc properties so the higher level code is not littered with bit stuffing.


Have fun.

Report message to a moderator

Re: Need to move away from using names as IDs Sat, 25 June 2011 23:15 Go to previous messageGo to next message
Aeglos is currently offline Aeglos

 
Chief Warrant Officer 1
Stars! Nova developer
Stars! Nova developer

Messages: 142
Registered: May 2011
Location: Chile
Key is already in Item, just needs modification.

I suggest we name the empire Id as Key also, to not get redundant "empiredata.empireid" variables, but simply "empiredata.Key". or to name both as Id instead.

DesignId, FleetId, and DesignId should be named as such.

We should also convert all .Owner properties of variables to be ints and reference to the empire Key.

Report message to a moderator

Re: Need to move away from using names as IDs Mon, 27 June 2011 05:49 Go to previous messageGo to next message
Aeglos is currently offline Aeglos

 
Chief Warrant Officer 1
Stars! Nova developer
Stars! Nova developer

Messages: 142
Registered: May 2011
Location: Chile
I support the Int keys btw. When I argued of the xml readability I was thinking of the general playerbase, but they shouldn't be messing with their files anyway Twisted Evil

Let's abstract all the bit operations in Properties though, like, religiously. Those things can get messy in the middle of the code.


[Updated on: Mon, 27 June 2011 05:53]

Report message to a moderator

Re: Need to move away from using names as IDs Mon, 27 June 2011 05:51 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
Now i'm confused..more than one key?

Taking fleets as an example:

Fleets have a Key (string like "silicanoids/10") a FleetID (int - 10) and an Owner (string silicanoids).

We only need 2 of these don't we? The Owner and a Unique ID (fleetID or Key)

I see that "Key" is actually an Item property that gets overriden. However looking for uses I can't see why this is on the base class or why it would need to be unique for all items? However if it is then we could use this and prefix it with (e.g.) "F" for a fleet or "D" for a design.

Otherwise let's lose Key and use FleetId, DesignId etc.

As mentioned in the other thread as Client and Server need to allocate Fleet IDs these could be:

"silicanoids/s1" or "silicanoids/c1" with or without the "F" in front

If we move to an EmpireId then owner can become Empire and whatever that is would replace silicanoids in the id.

Report message to a moderator

Re: Need to move away from using names as IDs Mon, 27 June 2011 06:04 Go to previous messageGo to next message
Aeglos is currently offline Aeglos

 
Chief Warrant Officer 1
Stars! Nova developer
Stars! Nova developer

Messages: 142
Registered: May 2011
Location: Chile
Edited my last post, it should have read "properties", not "keys".

Let's consolidate the specs until now:

* Empire shall have a unique Id
* 'Owner' variable on all objects that require it shall match it's owning Empire's Id.
* Empire shall have three counters with configurable upper bounds; FleetCount, DesignCount, StarbaseDesignCount.
* Fleets shall have a unique FleetId composed of EmpireId+FleetCount.
* Designs shall have a unique DesignId composed of EmpireId+DesignCount.
* Starbase Designs shall have a unique DesignId composed of EmpireId+StarbaseDesignCount.
* All Ids shall be Ints.
* All Id calculations shall be abstracted in Properties

We might require to add prefixes as you said, for design separation between ships and starbases. Or we could begin the SB Ids at the normal designs upper bound.

And for clarity, let's drop the "Key" nomenclature and call them all Ids from now. It's getting confusing. In fact, I would vote that the properties are also all named Id; ie. Fleet.Id, Design.Id, Empire.Id. Looks more consistent and clear.


[Updated on: Mon, 27 June 2011 06:06]

Report message to a moderator

Re: Need to move away from using names as IDs Mon, 27 June 2011 06:18 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
Aeglos wrote on Mon, 27 June 2011 06:04

Edited my last post, it should have read "properties", not "keys".

Let's consolidate the specs until now:
<see above>



Yes

Report message to a moderator

Re: Need to move away from using names as IDs Mon, 27 June 2011 07:09 Go to previous messageGo to next message
Daniel is currently offline Daniel

 
Chief Warrant Officer 3
Stars! Nova developer
Stars! Nova developer

Messages: 179
Registered: April 2006
Location: Nowra, Australia
Aeglos wrote on Mon, 27 June 2011 20:04

Edited my last post, it should have read "properties", not "keys".

Let's consolidate the specs until now:

* Empire shall have a unique Id
* 'Owner' variable on all objects that require it shall match it's owning Empire's Id.
* Empire shall have three counters with configurable upper bounds; FleetCount, DesignCount, StarbaseDesignCount.
* Fleets shall have a unique FleetId composed of EmpireId+FleetCount.
* Designs shall have a unique DesignId composed of EmpireId+DesignCount.
* Starbase Designs shall have a unique DesignId composed of EmpireId+StarbaseDesignCount.
* All Ids shall be Ints.
* All Id calculations shall be abstracted in Properties

We might require to add prefixes as you said, for design separation between ships and starbases. Or we could begin the SB Ids at the normal designs upper bound.

And for clarity, let's drop the "Key" nomenclature and call them all Ids from now. It's getting confusing. In fact, I would vote that the properties are also all named Id; ie. Fleet.Id, Design.Id, Empire.Id. Looks more consistent and clear.


Sounds good to me.



Have fun.

Report message to a moderator

Re: Need to move away from using names as IDs Mon, 27 June 2011 07:22 Go to previous messageGo to next message
FurFuznel is currently offline FurFuznel

 
Lt. Junior Grade
Stars! Nova developer
Stars! Nova developer

Messages: 437
Registered: November 2002
Location: New Brunswick, Canada
Sounds good to me as well! Yeah for clarity! Sun is out


Shadallark <==> FurFuznel
Mental anguish is for those who choose to think - FurFuznel
running Mac OS X 10.6.7

Report message to a moderator

icon14.gif  Re: Need to move away from using names as IDs Mon, 27 June 2011 13:42 Go to previous messageGo to next message
evild00d is currently offline evild00d

 
Crewman 2nd Class
Stars! Nova developer
Stars! Nova developer

Messages: 14
Registered: July 2009
Location: Norway
Sounds good

+1

Report message to a moderator

Re: Need to move away from using names as IDs Tue, 28 June 2011 15:11 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
So the next question is Who?

My first thought was that I need to sort FleetIDs out for the split/merge - so this would mean removing .Key from Item and making the relevant Id properties that are needed.

However we don't (I believe) have the Empire concept yet. Rather than do that right now a "Race" could be assigned an ID on new game creation and we can then use that in the other Ids for now.

Then later we can create Empire, such that each Empire has a link to a Race (so Empire's can share a Race). This will involve changing everything that uses Race.Name to Empire.Name, but use the Id as a key (which I would do by renaming Race.Name to Name2 and see what breaks!)

This was gets us a release with more features quicker. Or I can volunteer to just do the whole hog first?

Report message to a moderator

Re: Need to move away from using names as IDs Tue, 28 June 2011 15:27 Go to previous messageGo to next message
FurFuznel is currently offline FurFuznel

 
Lt. Junior Grade
Stars! Nova developer
Stars! Nova developer

Messages: 437
Registered: November 2002
Location: New Brunswick, Canada
If you are willing to take on the task, Whip or work with someone to take on the task I say we make the change now. Significant changes such as this, which will have far reaching implications and cause many dramatic improvements are good to do sooner than later.

The work I am doing on the Production Dialogue will not be effected either way.



Shadallark <==> FurFuznel
Mental anguish is for those who choose to think - FurFuznel
running Mac OS X 10.6.7

Report message to a moderator

Re: Need to move away from using names as IDs Tue, 28 June 2011 18:42 Go to previous messageGo to next message
Aeglos is currently offline Aeglos

 
Chief Warrant Officer 1
Stars! Nova developer
Stars! Nova developer

Messages: 142
Registered: May 2011
Location: Chile
I can work on this too. There are things that need the Ids to be fixed with the new intel objects anyways and I have nothing of more priority on my to-do list for now.

I say we skip adding a race id and go right away to empire level and fix things right away.

If you want to volunteer for the whole thing Mus to avoid SVN conflicts that may arise if we both do it, then I'll work on something else, like the scanning or server structure. Or if you want to tackle something else in the meanwhile, I can do it myself too.

One thing: Right now the empires are created as a by product of players... that is, one per player in the game. Let's leave it like that for now and focus on the Ids. We can change from players to empires as the primary game structure later.


[Updated on: Tue, 28 June 2011 18:44]

Report message to a moderator

Re: Need to move away from using names as IDs Wed, 29 June 2011 02:06 Go to previous messageGo to next message
Aeglos is currently offline Aeglos

 
Chief Warrant Officer 1
Stars! Nova developer
Stars! Nova developer

Messages: 142
Registered: May 2011
Location: Chile
I commited r669 in which I implemented Id properties for EmpireData and Item. Also Owner property for Item which I left commented for now due to conflict with the Owner string variable; refactoring pending.

Empire.Id should be set on game creation only from an int 0-127. It internally left-shifts into a range of 0x01000000 to 0x7F000000.

Item.Id gets or sets the lower 24 bits of it's private itemId variable. As Mus suggested that gives us over 16 million unique Ids per empire.

Item.Owner gets or sets the upper 7 bits of itemId, and it should be set from it's owning empire's Id.

The only concern I have is that design Ids can match fleet ids... for example fleet 1 and design 1 will have the same Id on the same empire. It shouldn't cause any issues though, since those are never on the same collection and there's the Type variable to distinguish them (which, btw, we should migrate to an enum).

So next step, migrate to the new Ids. We can probably work on that at the same time without too much hassle. EmpireData still needs to be assigned an Id, and needs it's counters. I'll work on that now.

EDIT: Empires are assigned Ids based on playernumber for now. EmpireData now contains counters for fleets and designs, and I added some global ints to define limits for them (need to be moved to the new game wizard).

I have a concern though: int counters have a problem. When a fleet or design is deleted, we cant simply reduce the counter by one; we would also need to re-index all designs and fleets. We need a way to tag available numbers on the counters. Perhaps use a dictionary<int, bool> with with <item.id, isUsed> instead of plain ints?

EDIT2: Working on applying Owner property now.


[Updated on: Wed, 29 June 2011 04:19]

Report message to a moderator

Re: Need to move away from using names as IDs Wed, 29 June 2011 05:13 Go to previous messageGo to next message
FurFuznel is currently offline FurFuznel

 
Lt. Junior Grade
Stars! Nova developer
Stars! Nova developer

Messages: 437
Registered: November 2002
Location: New Brunswick, Canada
Aeglos wrote on Wed, 29 June 2011 03:06

I have a concern though: int counters have a problem. When a fleet or design is deleted, we cant simply reduce the counter by one; we would also need to re-index all designs and fleets. We need a way to tag available numbers on the counters. Perhaps use a dictionary<int, bool> with with <item.id, isUsed> instead of plain ints?

If possible, I would recommend avoiding the re-use of IDs during the same turn. I realize this will make the server's processing (and some of the client's processing) of turns more difficult, but I always found it annoying in Stars! that the messages would refer to a fleet but clicking on the GoTo button did not show me that fleet but some other fleet that was now assigned the referenced fleet id.

There might not be an easy way to store IDs that will be freed up at the end of the turn and then free them up as the last step in the processing. If it would be too difficult to implement this I understand.



Shadallark <==> FurFuznel
Mental anguish is for those who choose to think - FurFuznel
running Mac OS X 10.6.7

Report message to a moderator

Re: Need to move away from using names as IDs Wed, 29 June 2011 06:10 Go to previous messageGo to next message
Musmuris

 
Master Chief Petty Officer
Stars! Nova developer
Stars! Nova developer

Messages: 96
Registered: June 2011
I would never re-use an ID. A new fleet gets a the next ID. When one is deleted it's just gone and removed from the AllFleets dictionary etc.

Report message to a moderator

Re: Need to move away from using names as IDs Wed, 29 June 2011 06:29 Go to previous messageGo to next message
Aeglos is currently offline Aeglos

 
Chief Warrant Officer 1
Stars! Nova developer
Stars! Nova developer

Messages: 142
Registered: May 2011
Location: Chile
But what do we do when the counter runs out?

The standard Stars! limits are 16 ship designs and 10 starbase designs, while those are going to be configurable, they should default to Stars! values for balance reasons as stated on the wiki. And even if they don't, there is still a chance they may exhaust.

Although, we could just use those for Ids and track the total amounts with Count from their collections. Might be a better idea.

On a related note, I migrated most of the code to use the Owner property. Might commit in the next hour or so.


[Updated on: Wed, 29 June 2011 06:34]

Report message to a moderator

Re: Need to move away from using names as IDs Wed, 29 June 2011 06:54 Go to previous messageGo to previous message
Aeglos is currently offline Aeglos

 
Chief Warrant Officer 1
Stars! Nova developer
Stars! Nova developer

Messages: 142
Registered: May 2011
Location: Chile
Well I commited now, as I need to get some sleep Hit over head

Item.Owner property is now used everywhere, as is Empire.Id. There is a Global.AllEmpires constant if something adresses everybody, like messages (It's set to 0 right now, so fleets and designs should start from 1 and reserve the 0 id for this special case) and another Global.NoOnwer (0 too, they are aliases) constant for things like unhabited stars.

What remains to do is to assign a proper Id to fleets and designs, and sort collections by Ids as I only changed AllEmpires and AllDesigns from the ServerState.

There will be plenty of GUI errors, as before this the GUI displayed race names through the Item.Owner string, and now it will display an ugly Hex value, if at all. In fact, the game is not even playable right now, as collections are indexed even worse than before. At most you can load the game and generate turns with no orders.

We need to sort out some client intel to something like EmpireReports or MetEmpires, so the client can locate their race names.

I think the sanest thing to do is send each client an aditional collection of EmpireData objects, which contain information from the other empires. Or perhaps a new slimmer object EmpireReport which contains their Id, Icon, RaceName and known designs... then we get rid of RaceIcons, AllEmpireIds and AllDesigns from the Intel object, and EnemyDesigns from the ClientState.

But for now, sleep Sleeping ... this restructuring turned out to be damn long and messy indeed.

Report message to a moderator

Previous Topic: Turn year
Next Topic: Orders overwriting Star Data
Goto Forum:
  


Current Time: Thu Mar 28 06:24:32 EDT 2024