This has been known for some time. IIRC there was a project to build a .m file sanitiser that removes that information (along with a bunch of other supposedly-invisible information) from .m files sent out by SAH; I don't know whether it was actually implemented or not.

EDIT: Everyone in the know was being a bit cagey about it, but some comments I read about this when it was first broached a few years back might suggest that among the other supposedly-invisible information in .m files is the location and stats of all fleets and planets (i.e. that the "can you see this" algorithm is client-side). If that's true, it makes for a VERY strong incentive not to permit out-of-client data extraction in ordinary games, because it would turn the game into a complete farce. Certainly, there's at least something bigger than the MT at stake here.

I would strongly object to the public dissemination of a tool to cheat in this fashion until we've at least discussed this a little more and determined exactly what is at stake, as well as the status of the sanitiser.

Here goes away another interesting dilemma we faced in a game ...

I therefore propose to add this exploit to the "Player Exploitable bugs / Features"-thread with the default setting "allowed". And then we make a tool so that it is easy for everyone to check out what the trader brings. in fact, I am working to include this in my GalaxyViewer-tool right now.

[...}as well as the status of the sanitiser.

magic9mushroom wrote on Fri, 05 August 2016 20:22

This has been known for some time. IIRC there was a project to build a .m file sanitiser that removes that information (along with a bunch of other supposedly-invisible information) from .m files sent out by SAH; I don't know whether it was actually implemented or not.

EDIT: Everyone in the know was being a bit cagey about it, but some comments I read about this when it was first broached a few years back might suggest that among the other supposedly-invisible information in .m files is the location and stats of all fleets and planets (i.e. that the "can you see this" algorithm is client-side). If that's true, it makes for a VERY strong incentive not to permit out-of-client data extraction in ordinary games, because it would turn the game into a complete farce. Certainly, there's at least something bigger than the MT at stake here.

I would strongly object to the public dissemination of a tool to cheat in this fashion until we've at least discussed this a little more and determined exactly what is at stake, as well as the status of the sanitiser.

I have already made an addition to my GalaxyViewer tool that shows what the MT brings. I will postpone committing it and pushing it to github until it has been discussed further. So until this has been properly discussed, I guess we with the knowhow should just try to restrain ourselves from checking what the MT brings, and that also includes using the already available GameToTestBed-tool for meeting the MT. Who was working on the m-file sanitizer? What else was to be included? (I myself started some work on an x-file sanitizer earlier this year, but because I had underestimated how it should respond to the minefield bug I haven't looked at it in a while)
However, I don't think the m-file contains normal info on what you do't see. If that was the case, the m-file-merger from a year back would have been pointless.

iztok wrote on Fri, 05 August 2016 11:39

Here goes away another interesting dilemma we faced in a game ...

Yes.

platon79 wrote on Fri, 05 August 2016 07:19

I therefore propose to add this exploit to the "Player Exploitable bugs / Features"-thread with the default setting "allowed". And then we make a tool so that it is easy for everyone to check out what the trader brings. in fact, I am working to include this in my GalaxyViewer-tool right now.

Better to put it into the open...
And thus I agree with your suggestions.
And I think that asap would be best. A direct link to the tool from the "Player Exploitable bugs / Features"-thread should put everybody on equal footing.

magic9mushroom wrote on Sat, 06 August 2016 02:22

[...}as well as the status of the sanitiser.

If there will be a sanitiser, we can use it. Until then... see above.

Of course, now I'm wondering how to reconcile that statement with the fact that platon79 clearly managed to hack his .m file productively. Was this game on SAH?

Ron does have a point that there is no way to defend player-hosted games (truly player-hosted, I mean, not "hosted" on SAH) from exploitation. That's the expressed reason why the details of this have never been made public.

I have now tried to PM Ron about the status of the m-file sanitizer/scrubber. Hopefully he will reply soon.

platon79 wrote on Thu, 11 August 2016 13:39

I have now tried to PM Ron about the status of the m-file sanitizer/scrubber. Hopefully he will reply soon.

Yes, I have a StarsKnowledgeCleaner.exe file that can sanitize individual .m files or a folder of .m files.

Xyligun said "I've made more tests and it seems that situation is much better with that hidden knowledge in jrc4 than in jrc3. In jrc3 it works exactly like I described earlier and puts all that hidden knowledge into CA m file. In jrc4 it still puts this knowledge block, but it zeroed it the same way as cleaner. So, the only available hidden info in jrc4 is info about traders and wormholes."

But since only traders and wormhole info is in the .m files for Jrc4 games, is this utility still needed?

platon79 wrote on Fri, 05 August 2016 07:19

I therefore propose to add this exploit to the "Player Exploitable bugs / Features"-thread with the default setting "allowed". And then we make a tool so that it is easy for everyone to check out what the trader brings. in fact, I am working to include this in my GalaxyViewer-tool right now.

Better to put it into the open...
And thus I agree with your suggestions.
And I think that asap would be best. A direct link to the tool from the "Player Exploitable bugs / Features"-thread should put everybody on equal footing.

magic9mushroom wrote on Sat, 06 August 2016 02:22

...as well as the status of the sanitiser.

If there will be a sanitiser, we can use it. Until then... see above.

While playing hosts of private games would still be able to cheat, that can no longer be averted.

Ron wrote on Fri, 12 August 2016 13:06

platon79 wrote on Thu, 11 August 2016 13:39

I have now tried to PM Ron about the status of the m-file sanitizer/scrubber. Hopefully he will reply soon.

Yes, I have a StarsKnowledgeCleaner.exe file that can sanitize individual .m files or a folder of .m files.

Xyligun said "I've made more tests and it seems that situation is much better with that hidden knowledge in jrc4 than in jrc3. In jrc3 it works exactly like I described earlier and puts all that hidden knowledge into CA m file. In jrc4 it still puts this knowledge block, but it zeroed it the same way as cleaner. So, the only available hidden info in jrc4 is info about traders and wormholes."

But since only traders and wormhole info is in the .m files for Jrc4 games, is this utility still needed?

Well, it's clearly needed if we're to have "no hacking allowed" games, as (as NH and others have correctly noted) there's no other way to enforce it. Certainly, games without info-hacking banned seem plausible given this new data, but I for one have little interest in playing them.

Would it be possible to publically release the sanitiser? Given that a data-extractor utility is about to be released, it would seem there's little remaining benefit in security-through-obscurity. While playing hosts of private games would still be able to cheat, that can no longer be averted.

Agreed. 100%. I can only imagine the motive to fight transparency belonging to players that intend to use the exploit and don't want competition.

So Ron, do you plan to implement your exe-file to the genning routine if enough players think that the MT-info and wormhole info should stay unrevealed?

Perhaps it should be made optional on a per-game basis. Let the players/host decide beforehand to use it or not.

m.a@stars wrote on Fri, 12 August 2016 18:16

Perhaps it should be made optional on a per-game basis. Let the players/host decide beforehand to use it or not.

I consider that ideal, if possible and easily implemented.